2023-12-17T11:30:00.167 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-17T11:30:00.167 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} --verbose --name rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156 -- /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/orig.config.yaml 2023-12-17T11:30:00.221 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-17T11:30:00.326 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156 branch: rishabh-2023dec11 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7495156' kernel: flavor: default kdb: true sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-2023dec11 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: a62fda73efba5520d409bb5a21bd8373289dda5d subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: a62fda73efba5520d409bb5a21bd8373289dda5d kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2023dec11 sha1: a62fda73efba5520d409bb5a21bd8373289dda5d owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5791 sha1: a62fda73efba5520d409bb5a21bd8373289dda5d sleep_before_teardown: 0 subset: 209/512 suite: fs suite_branch: rishabh-2023dec11 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a62fda73efba5520d409bb5a21bd8373289dda5d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJfU3CHyQpUsQAcUcyVukX5f4b233EJ0cgE5MeH0dQyPmvyV0T2YL2GB9Uh6UH/cMITD1X+OOZLl6TZrR/V9l4= smithi141.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOe8Da5pv1SEaTva5QR3wtOn+H7+J1X6wXZUHaF/ec9BRlBZnGoS14lnzlLQRXCyLk4pSvIsiug/Dz1EjWQEVg0= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMO8E8LJa+RODPFtq6V0DO8UTBc3wGWc8fzkgc9tgZJQ2rcpwfOjAWCqFj7QqTSQFuD1xjwc9S3RUJZU2bHW8Q= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-17_11:19:43 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078 2023-12-17T11:30:00.326 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa; will attempt to use it 2023-12-17T11:30:00.327 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks 2023-12-17T11:30:00.327 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-17T11:30:00.329 INFO:teuthology.task.internal:Checking packages... 2023-12-17T11:30:00.357 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a62fda73efba5520d409bb5a21bd8373289dda5d' 2023-12-17T11:30:00.358 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-17T11:30:00.358 INFO:teuthology.packaging:ref: None 2023-12-17T11:30:00.358 INFO:teuthology.packaging:tag: None 2023-12-17T11:30:00.358 INFO:teuthology.packaging:branch: rishabh-2023dec11 2023-12-17T11:30:00.358 INFO:teuthology.packaging:sha1: a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:30:00.358 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-2023dec11 2023-12-17T11:30:00.576 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-119.ga62fda73 2023-12-17T11:30:00.577 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-17T11:30:00.587 INFO:teuthology.task.internal:no buildpackages task found 2023-12-17T11:30:00.587 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-17T11:30:00.592 INFO:teuthology.task.internal:Saving configuration 2023-12-17T11:30:00.612 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-17T11:30:00.618 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-17T11:30:00.644 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-17 11:21:20.059199', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJfU3CHyQpUsQAcUcyVukX5f4b233EJ0cgE5MeH0dQyPmvyV0T2YL2GB9Uh6UH/cMITD1X+OOZLl6TZrR/V9l4='} 2023-12-17T11:30:00.665 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi141.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-17 11:21:20.058187', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOe8Da5pv1SEaTva5QR3wtOn+H7+J1X6wXZUHaF/ec9BRlBZnGoS14lnzlLQRXCyLk4pSvIsiug/Dz1EjWQEVg0='} 2023-12-17T11:30:00.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-17 11:21:20.060050', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMO8E8LJa+RODPFtq6V0DO8UTBc3wGWc8fzkgc9tgZJQ2rcpwfOjAWCqFj7QqTSQFuD1xjwc9S3RUJZU2bHW8Q='} 2023-12-17T11:30:00.688 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-17T11:30:00.694 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-12-17T11:30:00.694 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-12-17T11:30:00.694 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-12-17T11:30:00.694 INFO:teuthology.run_tasks:Running task console_log... 2023-12-17T11:30:00.821 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f63ce616dc0>, signals=[15]) 2023-12-17T11:30:00.821 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-17T11:30:00.827 INFO:teuthology.task.internal:Opening connections... 2023-12-17T11:30:00.828 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2023-12-17T11:30:00.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:30:00.898 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi141.front.sepia.ceph.com 2023-12-17T11:30:00.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:30:00.967 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2023-12-17T11:30:00.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:30:01.037 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-17T11:30:01.043 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2023-12-17T11:30:01.064 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2023-12-17T11:30:01.065 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="Red Hat Enterprise Linux" 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8.6 (Ootpa)" 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:ID="rhel" 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="fedora" 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8.6" 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8" 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2023-12-17T11:30:01.125 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-17T11:30:01.126 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://www.redhat.com/" 2023-12-17T11:30:01.126 INFO:teuthology.orchestra.run.smithi027.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-17T11:30:01.126 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-17T11:30:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:30:01.126 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-17T11:30:01.126 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-17T11:30:01.126 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-17T11:30:01.126 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-17T11:30:01.127 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-12-17T11:30:01.151 DEBUG:teuthology.orchestra.run.smithi141:> uname -m 2023-12-17T11:30:01.173 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64 2023-12-17T11:30:01.174 DEBUG:teuthology.orchestra.run.smithi141:> cat /etc/os-release 2023-12-17T11:30:01.233 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="Red Hat Enterprise Linux" 2023-12-17T11:30:01.233 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8.6 (Ootpa)" 2023-12-17T11:30:01.233 INFO:teuthology.orchestra.run.smithi141.stdout:ID="rhel" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="fedora" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8.6" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://www.redhat.com/" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-17T11:30:01.234 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-17T11:30:01.235 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2023-12-17T11:30:01.259 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2023-12-17T11:30:01.280 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2023-12-17T11:30:01.281 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2023-12-17T11:30:01.339 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="Red Hat Enterprise Linux" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="8.6 (Ootpa)" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:ID="rhel" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="fedora" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="8.6" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el8" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.redhat.com/" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:30:01.340 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-17T11:30:01.341 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-17T11:30:01.341 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-17T11:30:01.341 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-17T11:30:01.341 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2023-12-17T11:30:01.358 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-17T11:30:01.369 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-17T11:30:01.392 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-17T11:30:01.393 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2023-12-17T11:30:01.395 DEBUG:teuthology.orchestra.run.smithi141:> test '!' -e /home/ubuntu/cephtest 2023-12-17T11:30:01.397 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2023-12-17T11:30:01.415 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-17T11:30:01.421 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-17T11:30:01.421 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2023-12-17T11:30:01.455 DEBUG:teuthology.orchestra.run.smithi141:> test -z $(ls -A /var/lib/ceph) 2023-12-17T11:30:01.457 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2023-12-17T11:30:01.483 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-17T11:30:01.558 INFO:teuthology.run_tasks:Running task kernel... 2023-12-17T11:30:01.572 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'} 2023-12-17T11:30:01.573 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}}, timeout 300 2023-12-17T11:30:01.573 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-17T11:30:01.573 DEBUG:teuthology.orchestra.run.smithi141:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-17T11:30:01.574 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-17T11:30:01.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T11:30:01.592 DEBUG:teuthology.task.kernel:sha1 for mon.c is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.592 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-17T11:30:01.593 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2023-12-17T11:30:01.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T11:30:01.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T11:30:01.595 DEBUG:teuthology.task.kernel:sha1 for mon.a is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.595 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-17T11:30:01.595 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-12-17T11:30:01.596 DEBUG:teuthology.task.kernel:sha1 for mon.b is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.596 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-17T11:30:01.596 DEBUG:teuthology.orchestra.run.smithi141:> uname -r 2023-12-17T11:30:01.651 INFO:teuthology.orchestra.run.smithi192.stdout:4.18.0-513.9.1.el8_9.x86_64 2023-12-17T11:30:01.652 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.652 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-17T11:30:01.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.659 INFO:teuthology.orchestra.run.smithi027.stdout:4.18.0-513.9.1.el8_9.x86_64 2023-12-17T11:30:01.659 INFO:teuthology.orchestra.run.smithi141.stdout:4.18.0-513.9.1.el8_9.x86_64 2023-12-17T11:30:01.659 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.659 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-17T11:30:01.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.661 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.661 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-17T11:30:01.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:01.879 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2023-12-17T11:30:01.880 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum upgrade -y linux-firmware 2023-12-17T11:30:01.893 DEBUG:teuthology.orchestra.run.smithi141:> sudo yum upgrade -y linux-firmware 2023-12-17T11:30:03.357 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-12-17T11:30:03.357 INFO:teuthology.orchestra.run.smithi141.stdout:Unable to read consumer identity 2023-12-17T11:30:03.465 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-12-17T11:30:03.465 INFO:teuthology.orchestra.run.smithi192.stdout:Unable to read consumer identity 2023-12-17T11:30:03.534 INFO:teuthology.orchestra.run.smithi027.stdout:Updating Subscription Management repositories. 2023-12-17T11:30:03.534 INFO:teuthology.orchestra.run.smithi027.stdout:Unable to read consumer identity 2023-12-17T11:30:03.663 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:30:03.663 INFO:teuthology.orchestra.run.smithi141.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-17T11:30:03.664 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:30:03.689 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:30:03.689 INFO:teuthology.orchestra.run.smithi192.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-17T11:30:03.690 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:30:03.804 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:30:03.804 INFO:teuthology.orchestra.run.smithi027.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-17T11:30:03.804 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:30:05.577 INFO:teuthology.orchestra.run.smithi192.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-12-17T11:30:05.624 INFO:teuthology.orchestra.run.smithi141.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-12-17T11:30:05.718 INFO:teuthology.orchestra.run.smithi027.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-12-17T11:30:06.450 INFO:teuthology.orchestra.run.smithi141.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-12-17T11:30:06.902 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-12-17T11:30:09.795 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 3.8 MB/s | 16 MB 00:04 2023-12-17T11:30:10.017 INFO:teuthology.orchestra.run.smithi141.stdout:lab-extras 429 kB/s | 24 kB 00:00 2023-12-17T11:30:10.669 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 438 kB/s | 24 kB 00:00 2023-12-17T11:30:10.809 INFO:teuthology.orchestra.run.smithi141.stdout:Dependencies resolved. 2023-12-17T11:30:10.810 INFO:teuthology.orchestra.run.smithi141.stdout:Nothing to do. 2023-12-17T11:30:10.810 INFO:teuthology.orchestra.run.smithi141.stdout:Complete! 2023-12-17T11:30:10.959 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.b... 2023-12-17T11:30:10.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:11.275 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ 2023-12-17T11:30:11.275 DEBUG:teuthology.orchestra.run.smithi141:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-17T11:30:11.277 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi141.front.sepia.ceph.com to complete... 2023-12-17T11:30:11.538 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-12-17T11:30:11.539 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2023-12-17T11:30:11.539 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-12-17T11:30:11.661 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.a... 2023-12-17T11:30:11.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:11.803 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ 2023-12-17T11:30:11.804 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-17T11:30:11.806 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2023-12-17T11:30:13.410 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 406 kB/s | 24 kB 00:00 2023-12-17T11:30:14.222 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-12-17T11:30:14.223 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2023-12-17T11:30:14.223 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-12-17T11:30:14.375 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.c... 2023-12-17T11:30:14.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-17T11:30:14.673 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ 2023-12-17T11:30:14.673 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-17T11:30:14.676 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi192.front.sepia.ceph.com to complete... 2023-12-17T11:34:00.123 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17 11:34:00 URL:https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [383180176/383180176] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-17T11:34:00.123 INFO:teuthology.orchestra.run.smithi192.stderr:FINISHED --2023-12-17 11:34:00-- 2023-12-17T11:34:00.123 INFO:teuthology.orchestra.run.smithi192.stderr:Total wall clock time: 3m 45s 2023-12-17T11:34:00.123 INFO:teuthology.orchestra.run.smithi192.stderr:Downloaded: 1 files, 365M in 3m 45s (1.62 MB/s) 2023-12-17T11:34:00.127 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.c... 2023-12-17T11:34:00.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-17T11:34:01.891 INFO:teuthology.orchestra.run.smithi192.stdout:Verifying... ######################################## 2023-12-17T11:34:01.981 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing... ######################################## 2023-12-17T11:34:02.129 INFO:teuthology.orchestra.run.smithi192.stdout:Updating / installing... 2023-12-17T11:34:40.880 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-17T11:34:45.515 INFO:teuthology.orchestra.run.smithi192.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-17T11:35:04.997 INFO:teuthology.orchestra.run.smithi192.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-17T11:36:02.873 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17 11:36:02 URL:https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [383180176/383180176] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-17T11:36:02.873 INFO:teuthology.orchestra.run.smithi141.stderr:FINISHED --2023-12-17 11:36:02-- 2023-12-17T11:36:02.873 INFO:teuthology.orchestra.run.smithi141.stderr:Total wall clock time: 5m 52s 2023-12-17T11:36:02.873 INFO:teuthology.orchestra.run.smithi141.stderr:Downloaded: 1 files, 365M in 5m 51s (1.04 MB/s) 2023-12-17T11:36:02.876 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.b... 2023-12-17T11:36:02.877 DEBUG:teuthology.orchestra.run.smithi141:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-17T11:36:04.672 INFO:teuthology.orchestra.run.smithi141.stdout:Verifying... ######################################## 2023-12-17T11:36:04.752 INFO:teuthology.orchestra.run.smithi141.stdout:Preparing... ######################################## 2023-12-17T11:36:04.916 INFO:teuthology.orchestra.run.smithi141.stdout:Updating / installing... 2023-12-17T11:36:43.164 INFO:teuthology.orchestra.run.smithi141.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-17T11:36:44.567 INFO:teuthology.orchestra.run.smithi141.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-17T11:37:03.716 INFO:teuthology.orchestra.run.smithi141.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-17T11:37:09.991 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-17T11:37:09.992 DEBUG:teuthology.orchestra.run.smithi192:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-17T11:37:10.046 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.046 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.046 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.046 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-17T11:37:10.047 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-17T11:37:10.048 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-17T11:37:10.049 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-17T11:37:10.050 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-17T11:37:10.051 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-17T11:37:10.052 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-17T11:37:10.053 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-17T11:37:10.054 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-17T11:37:10.055 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-17T11:37:10.056 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-17T11:37:10.057 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-17T11:37:10.058 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-17T11:37:10.059 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-17T11:37:10.060 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-17T11:37:10.061 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-17T11:37:10.062 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-17T11:37:10.063 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-17T11:37:10.064 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-17T11:37:10.065 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-17T11:37:10.066 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-17T11:37:10.067 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-17T11:37:10.068 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-17T11:37:10.069 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-17T11:37:10.070 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-17T11:37:10.072 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-17T11:37:10.073 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-17T11:37:10.074 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-17T11:37:10.075 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-17T11:37:10.076 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-17T11:37:10.077 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-17T11:37:10.078 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-17T11:37:10.079 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-17T11:37:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-17T11:37:10.081 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-17T11:37:10.082 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-17T11:37:10.083 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-17T11:37:10.084 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-17T11:37:10.085 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-17T11:37:10.086 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-17T11:37:10.087 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-17T11:37:10.088 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-17T11:37:10.089 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-17T11:37:10.090 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-17T11:37:10.091 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-17T11:37:10.092 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-17T11:37:10.093 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-17T11:37:10.094 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-17T11:37:10.095 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-17T11:37:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-17T11:37:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-17T11:37:10.098 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-17T11:37:10.099 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-17T11:37:10.100 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-17T11:37:10.101 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-17T11:37:10.102 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-17T11:37:10.103 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-17T11:37:10.104 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-17T11:37:10.105 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-17T11:37:10.106 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-17T11:37:10.107 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-17T11:37:10.108 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-17T11:37:10.109 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-17T11:37:10.110 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-17T11:37:10.111 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-17T11:37:10.112 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-17T11:37:10.113 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-17T11:37:10.114 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-17T11:37:10.115 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-17T11:37:10.116 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-17T11:37:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-17T11:37:10.118 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-17T11:37:10.119 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-17T11:37:10.120 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-17T11:37:10.121 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-17T11:37:10.122 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-17T11:37:10.123 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-17T11:37:10.124 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-17T11:37:10.125 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-17T11:37:10.126 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-17T11:37:10.127 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-17T11:37:10.128 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-17T11:37:10.129 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-17T11:37:10.130 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-17T11:37:10.131 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-17T11:37:10.132 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-17T11:37:10.133 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-17T11:37:10.134 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-17T11:37:10.135 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-17T11:37:10.136 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-17T11:37:10.137 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-17T11:37:10.138 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-17T11:37:10.139 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-17T11:37:10.140 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-17T11:37:10.141 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-17T11:37:10.142 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-17T11:37:10.143 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-17T11:37:10.144 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-17T11:37:10.145 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-17T11:37:10.146 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-17T11:37:10.147 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-17T11:37:10.148 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-17T11:37:10.149 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-17T11:37:10.150 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-17T11:37:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-17T11:37:10.152 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-17T11:37:10.153 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-17T11:37:10.154 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-17T11:37:10.155 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-17T11:37:10.156 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-17T11:37:10.157 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-17T11:37:10.158 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-17T11:37:10.159 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-17T11:37:10.160 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-17T11:37:10.161 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-17T11:37:10.162 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-17T11:37:10.163 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-17T11:37:10.164 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-17T11:37:10.165 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-17T11:37:10.166 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-17T11:37:10.167 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-17T11:37:10.168 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-17T11:37:10.169 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-17T11:37:10.170 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-17T11:37:10.171 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-17T11:37:10.172 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-17T11:37:10.173 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-17T11:37:10.174 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-17T11:37:10.175 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-17T11:37:10.176 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-17T11:37:10.177 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-17T11:37:10.178 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-17T11:37:10.179 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-17T11:37:10.180 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-17T11:37:10.181 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.181 DEBUG:teuthology.orchestra.run.smithi192:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-17T11:37:10.204 INFO:teuthology.orchestra.run.smithi192.stdout:postinstall scriptlet (using /bin/sh): 2023-12-17T11:37:10.204 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-17T11:37:10.204 INFO:teuthology.orchestra.run.smithi192.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-17T11:37:10.205 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-17T11:37:10.206 INFO:teuthology.orchestra.run.smithi192.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.206 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2023-12-17T11:37:10.206 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2023-12-17T11:37:10.291 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2023-12-17T11:37:10.291 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2023-12-17T11:37:10.291 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.02 2023-12-17T11:37:10.291 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 123.el8 2023-12-17T11:37:10.291 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Group : System Environment/Base 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 9265450 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Relocations : (not relocatable) 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : Red Hat, Inc. 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : Red Hat, Inc. 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2023-12-17T11:37:10.292 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:37:10.293 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-17T11:37:10.293 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-17T11:37:10.293 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-17T11:37:10.293 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2023-12-17T11:37:10.293 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:37:10.293 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2023-12-17T11:37:10.294 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-17T11:37:10.294 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 6.6.0-gcd68252d7b4d 2023-12-17T11:37:10.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-17T11:37:11.063 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2023-12-17T11:37:12.989 INFO:teuthology.orchestra.run.smithi192.stderr:done 2023-12-17T11:37:12.991 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2023-12-17T11:37:13.008 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.ZOFqDut7JF 2023-12-17T11:37:13.008 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZOFqDut7JF 2023-12-17T11:37:13.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.ZOFqDut7JF 2023-12-17T11:37:13.245 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.ZOFqDut7JF is 6KB 2023-12-17T11:37:13.294 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.ZOFqDut7JF 2023-12-17T11:37:13.310 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2023-12-17T11:37:13.375 INFO:teuthology.orchestra.run.smithi192.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2023-12-17T11:37:13.376 INFO:teuthology.orchestra.run.smithi192.stdout:2db68f91c8f90721a6596efe657a5324-6.6.0-gcd68252d7b4d.conf 2023-12-17T11:37:13.376 INFO:teuthology.orchestra.run.smithi192.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2023-12-17T11:37:13.376 INFO:teuthology.orchestra.run.smithi192.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2023-12-17T11:37:13.376 INFO:teuthology.orchestra.run.smithi192.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2023-12-17T11:37:13.377 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.6.0-gcd68252d7b4d 2023-12-17T11:37:13.535 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2023-12-17T11:37:24.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17 11:37:24 URL:https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [383180176/383180176] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-17T11:37:24.957 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2023-12-17 11:37:24-- 2023-12-17T11:37:24.958 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 7m 13s 2023-12-17T11:37:24.958 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 365M in 7m 12s (865 KB/s) 2023-12-17T11:37:24.960 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.a... 2023-12-17T11:37:24.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-17T11:37:26.837 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2023-12-17T11:37:26.936 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2023-12-17T11:37:27.104 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2023-12-17T11:37:43.537 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2023-12-17T11:37:43.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-17T11:37:43.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:38:07.936 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-17T11:38:10.106 INFO:teuthology.orchestra.run.smithi027.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-17T11:38:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-17T11:38:43.541 DEBUG:teuthology.orchestra.remote:timed out 2023-12-17T11:38:52.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-17T11:38:52.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:39:00.135 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-12-17T11:39:00.540 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-17T11:39:00.541 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.6.0-gcd68252d7b4d"... 2023-12-17T11:39:00.541 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2023-12-17T11:39:00.583 INFO:teuthology.orchestra.run.smithi192.stdout:6.6.0-gcd68252d7b4d 2023-12-17T11:39:00.583 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-17T11:39:00.583 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-17T11:39:00.583 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-17T11:39:01.584 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2023-12-17T11:39:01.584 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-17T11:39:01.716 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2023-12-17T11:39:01.745 DEBUG:teuthology.parallel:result is None 2023-12-17T11:39:10.171 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi141.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-17T11:39:10.171 DEBUG:teuthology.orchestra.run.smithi141:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-17T11:39:10.225 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-17T11:39:10.226 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-17T11:39:10.227 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-17T11:39:10.228 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-17T11:39:10.229 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-17T11:39:10.230 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-17T11:39:10.231 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-17T11:39:10.232 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-17T11:39:10.233 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-17T11:39:10.234 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-17T11:39:10.235 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-17T11:39:10.236 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-17T11:39:10.237 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-17T11:39:10.238 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-17T11:39:10.239 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-17T11:39:10.240 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-17T11:39:10.241 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-17T11:39:10.242 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-17T11:39:10.243 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-17T11:39:10.244 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-17T11:39:10.245 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-17T11:39:10.246 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-17T11:39:10.247 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-17T11:39:10.248 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-17T11:39:10.249 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-17T11:39:10.250 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-17T11:39:10.251 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-17T11:39:10.252 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-17T11:39:10.253 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-17T11:39:10.254 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-17T11:39:10.255 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-17T11:39:10.256 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-17T11:39:10.257 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-17T11:39:10.258 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-17T11:39:10.259 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-17T11:39:10.260 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-17T11:39:10.261 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-17T11:39:10.262 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-17T11:39:10.263 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-17T11:39:10.264 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-17T11:39:10.265 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-17T11:39:10.266 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-17T11:39:10.267 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-17T11:39:10.268 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-17T11:39:10.269 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-17T11:39:10.270 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-17T11:39:10.271 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-17T11:39:10.272 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-17T11:39:10.273 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-17T11:39:10.274 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-17T11:39:10.275 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-17T11:39:10.276 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-17T11:39:10.277 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-17T11:39:10.278 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-17T11:39:10.279 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-17T11:39:10.280 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-17T11:39:10.281 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-17T11:39:10.282 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-17T11:39:10.283 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-17T11:39:10.284 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-17T11:39:10.285 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-17T11:39:10.286 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-17T11:39:10.287 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-17T11:39:10.288 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-17T11:39:10.289 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-17T11:39:10.290 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-17T11:39:10.291 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-17T11:39:10.292 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-17T11:39:10.293 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-17T11:39:10.294 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-17T11:39:10.295 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-17T11:39:10.296 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-17T11:39:10.297 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-17T11:39:10.298 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-17T11:39:10.299 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-17T11:39:10.300 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-17T11:39:10.301 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-17T11:39:10.302 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-17T11:39:10.303 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-17T11:39:10.304 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-17T11:39:10.305 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-17T11:39:10.306 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-17T11:39:10.307 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-17T11:39:10.308 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-17T11:39:10.309 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-17T11:39:10.310 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-17T11:39:10.311 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-17T11:39:10.312 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-17T11:39:10.313 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-17T11:39:10.314 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-17T11:39:10.315 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-17T11:39:10.316 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-17T11:39:10.317 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-17T11:39:10.318 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-17T11:39:10.319 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-17T11:39:10.320 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-17T11:39:10.321 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-17T11:39:10.324 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-17T11:39:10.324 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-17T11:39:10.324 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-17T11:39:10.324 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-17T11:39:10.325 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-17T11:39:10.326 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-17T11:39:10.327 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-17T11:39:10.328 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-17T11:39:10.329 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-17T11:39:10.330 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-17T11:39:10.331 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-17T11:39:10.332 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-17T11:39:10.333 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-17T11:39:10.334 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-17T11:39:10.335 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-17T11:39:10.336 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-17T11:39:10.337 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-17T11:39:10.338 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-17T11:39:10.339 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-17T11:39:10.340 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-17T11:39:10.341 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-17T11:39:10.342 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-17T11:39:10.343 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-17T11:39:10.344 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-17T11:39:10.345 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-17T11:39:10.346 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-17T11:39:10.347 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-17T11:39:10.348 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-17T11:39:10.349 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-17T11:39:10.350 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-17T11:39:10.351 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-17T11:39:10.352 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-17T11:39:10.353 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-17T11:39:10.354 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-17T11:39:10.355 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-17T11:39:10.356 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-17T11:39:10.357 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-17T11:39:10.358 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-17T11:39:10.359 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-17T11:39:10.360 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-17T11:39:10.361 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-17T11:39:10.362 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-17T11:39:10.363 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-17T11:39:10.364 INFO:teuthology.orchestra.run.smithi141.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-17T11:39:10.365 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.365 DEBUG:teuthology.orchestra.run.smithi141:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:postinstall scriptlet (using /bin/sh): 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:fi 2023-12-17T11:39:10.389 INFO:teuthology.orchestra.run.smithi141.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:fi 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.390 INFO:teuthology.orchestra.run.smithi141.stdout:fi 2023-12-17T11:39:10.390 DEBUG:teuthology.orchestra.run.smithi141:> sudo rpm -qi grub2-tools 2023-12-17T11:39:10.462 INFO:teuthology.orchestra.run.smithi141.stdout:Name : grub2-tools 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Epoch : 1 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Version : 2.02 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Release : 123.el8 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Architecture: x86_64 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Group : System Environment/Base 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Size : 9265450 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:License : GPLv3+ 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Relocations : (not relocatable) 2023-12-17T11:39:10.463 INFO:teuthology.orchestra.run.smithi141.stdout:Packager : Red Hat, Inc. 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:Vendor : Red Hat, Inc. 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:Summary : Support tools for GRUB. 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:Description : 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:hardware devices. 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:39:10.464 INFO:teuthology.orchestra.run.smithi141.stdout:This subpackage provides tools for support of all platforms. 2023-12-17T11:39:10.465 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-17T11:39:10.465 INFO:teuthology.task.kernel:Updating grub on smithi141 to boot 6.6.0-gcd68252d7b4d 2023-12-17T11:39:10.466 DEBUG:teuthology.orchestra.run.smithi141:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-17T11:39:11.491 INFO:teuthology.orchestra.run.smithi141.stderr:Generating grub configuration file ... 2023-12-17T11:39:14.157 INFO:teuthology.orchestra.run.smithi141.stderr:done 2023-12-17T11:39:14.159 DEBUG:teuthology.orchestra.run.smithi141:> mktemp 2023-12-17T11:39:14.176 INFO:teuthology.orchestra.run.smithi141.stdout:/tmp/tmp.RMrwwmn3CV 2023-12-17T11:39:14.176 DEBUG:teuthology.orchestra.run.smithi141:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RMrwwmn3CV 2023-12-17T11:39:14.248 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod 0666 /tmp/tmp.RMrwwmn3CV 2023-12-17T11:39:14.415 DEBUG:teuthology.orchestra.remote:smithi141:/tmp/tmp.RMrwwmn3CV is 6KB 2023-12-17T11:39:14.464 DEBUG:teuthology.orchestra.run.smithi141:> rm -fr /tmp/tmp.RMrwwmn3CV 2023-12-17T11:39:14.480 DEBUG:teuthology.orchestra.run.smithi141:> sudo /bin/ls /boot/loader/entries || true 2023-12-17T11:39:14.546 INFO:teuthology.orchestra.run.smithi141.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2023-12-17T11:39:14.546 INFO:teuthology.orchestra.run.smithi141.stdout:2db68f91c8f90721a6596efe657a5324-6.6.0-gcd68252d7b4d.conf 2023-12-17T11:39:14.546 INFO:teuthology.orchestra.run.smithi141.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2023-12-17T11:39:14.546 INFO:teuthology.orchestra.run.smithi141.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2023-12-17T11:39:14.546 INFO:teuthology.orchestra.run.smithi141.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2023-12-17T11:39:14.547 DEBUG:teuthology.orchestra.run.smithi141:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.6.0-gcd68252d7b4d 2023-12-17T11:39:14.698 DEBUG:teuthology.orchestra.run.smithi141:> sudo shutdown -r now 2023-12-17T11:39:44.701 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-17T11:39:44.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-12-17T11:39:44.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:40:38.483 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-17T11:40:38.484 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-17T11:40:38.539 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.539 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.539 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-17T11:40:38.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-17T11:40:38.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-17T11:40:38.539 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-17T11:40:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-17T11:40:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-17T11:40:38.542 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-17T11:40:38.543 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-17T11:40:38.544 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-17T11:40:38.545 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-17T11:40:38.546 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-17T11:40:38.547 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-17T11:40:38.548 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-17T11:40:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-17T11:40:38.550 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-17T11:40:38.551 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-17T11:40:38.552 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-17T11:40:38.553 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-17T11:40:38.554 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-17T11:40:38.555 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-17T11:40:38.556 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-17T11:40:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-17T11:40:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-17T11:40:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-17T11:40:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-17T11:40:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-17T11:40:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-17T11:40:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-17T11:40:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-17T11:40:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-17T11:40:38.566 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-17T11:40:38.567 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-17T11:40:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-17T11:40:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-17T11:40:38.570 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-17T11:40:38.571 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-17T11:40:38.572 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-17T11:40:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-17T11:40:38.574 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-17T11:40:38.575 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-17T11:40:38.576 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-17T11:40:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-17T11:40:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-17T11:40:38.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-17T11:40:38.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-17T11:40:38.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-17T11:40:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-17T11:40:38.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-17T11:40:38.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-17T11:40:38.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-17T11:40:38.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-17T11:40:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-17T11:40:38.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-17T11:40:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-17T11:40:38.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-17T11:40:38.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-17T11:40:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-17T11:40:38.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-17T11:40:38.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-17T11:40:38.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-17T11:40:38.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-17T11:40:38.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-17T11:40:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-17T11:40:38.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-17T11:40:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-17T11:40:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-17T11:40:38.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-17T11:40:38.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-17T11:40:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-17T11:40:38.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-17T11:40:38.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-17T11:40:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-17T11:40:38.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-17T11:40:38.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-17T11:40:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-17T11:40:38.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-17T11:40:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-17T11:40:38.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-17T11:40:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-17T11:40:38.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-17T11:40:38.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-17T11:40:38.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-17T11:40:38.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-17T11:40:38.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-17T11:40:38.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-17T11:40:38.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-17T11:40:38.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-17T11:40:38.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-17T11:40:38.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-17T11:40:38.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-17T11:40:38.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-17T11:40:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-17T11:40:38.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-17T11:40:38.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-17T11:40:38.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-17T11:40:38.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-17T11:40:38.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-17T11:40:38.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-17T11:40:38.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-17T11:40:38.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-17T11:40:38.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-17T11:40:38.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-17T11:40:38.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-17T11:40:38.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-17T11:40:38.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-17T11:40:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-17T11:40:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-17T11:40:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-17T11:40:38.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-17T11:40:38.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-17T11:40:38.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-17T11:40:38.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-17T11:40:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-17T11:40:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-17T11:40:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-17T11:40:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-17T11:40:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-17T11:40:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-17T11:40:38.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-17T11:40:38.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-17T11:40:38.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-17T11:40:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-17T11:40:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-17T11:40:38.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-17T11:40:38.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-17T11:40:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-17T11:40:38.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-17T11:40:38.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-17T11:40:38.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-17T11:40:38.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-17T11:40:38.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-17T11:40:38.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-17T11:40:38.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-17T11:40:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-17T11:40:38.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-17T11:40:38.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-17T11:40:38.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-17T11:40:38.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-17T11:40:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-17T11:40:38.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-17T11:40:38.675 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.675 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-17T11:40:38.699 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2023-12-17T11:40:38.699 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-17T11:40:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.701 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-12-17T11:40:38.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2023-12-17T11:40:38.776 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2023-12-17T11:40:38.776 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2023-12-17T11:40:38.776 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.02 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 123.el8 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Group : System Environment/Base 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 9265450 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Relocations : (not relocatable) 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : Red Hat, Inc. 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : Red Hat, Inc. 2023-12-17T11:40:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:40:38.778 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2023-12-17T11:40:38.779 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-17T11:40:38.779 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.6.0-gcd68252d7b4d 2023-12-17T11:40:38.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-17T11:40:39.790 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2023-12-17T11:40:42.037 INFO:teuthology.orchestra.run.smithi027.stderr:done 2023-12-17T11:40:42.038 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2023-12-17T11:40:42.060 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.0wWSn8ryug 2023-12-17T11:40:42.061 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0wWSn8ryug 2023-12-17T11:40:42.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.0wWSn8ryug 2023-12-17T11:40:42.289 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.0wWSn8ryug is 6KB 2023-12-17T11:40:42.337 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.0wWSn8ryug 2023-12-17T11:40:42.353 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2023-12-17T11:40:42.418 INFO:teuthology.orchestra.run.smithi027.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2023-12-17T11:40:42.418 INFO:teuthology.orchestra.run.smithi027.stdout:2db68f91c8f90721a6596efe657a5324-6.6.0-gcd68252d7b4d.conf 2023-12-17T11:40:42.418 INFO:teuthology.orchestra.run.smithi027.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2023-12-17T11:40:42.418 INFO:teuthology.orchestra.run.smithi027.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2023-12-17T11:40:42.418 INFO:teuthology.orchestra.run.smithi027.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2023-12-17T11:40:42.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.6.0-gcd68252d7b4d 2023-12-17T11:40:42.578 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2023-12-17T11:40:44.705 DEBUG:teuthology.orchestra.remote:timed out 2023-12-17T11:40:53.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-12-17T11:40:53.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:41:01.112 DEBUG:teuthology.orchestra.run.smithi141:> true 2023-12-17T11:41:01.515 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-12-17T11:41:01.516 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-gcd68252d7b4d"... 2023-12-17T11:41:01.516 DEBUG:teuthology.orchestra.run.smithi141:> uname -r 2023-12-17T11:41:01.558 INFO:teuthology.orchestra.run.smithi141.stdout:6.6.0-gcd68252d7b4d 2023-12-17T11:41:01.559 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-17T11:41:01.559 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-17T11:41:01.559 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-17T11:41:02.560 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-17T11:41:02.560 DEBUG:teuthology.orchestra.run.smithi141:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-17T11:41:02.693 INFO:teuthology.orchestra.run.smithi141.stdout:ttyS1 2023-12-17T11:41:02.719 DEBUG:teuthology.parallel:result is None 2023-12-17T11:41:12.581 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-17T11:41:12.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-17T11:41:12.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:41:47.154 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-12-17T11:41:56.154 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-17T11:41:56.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:41:59.217 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-12-17T11:42:11.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-17T11:42:11.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:42:14.289 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-12-17T11:42:29.290 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-17T11:42:29.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:42:32.369 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-12-17T11:42:50.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-17T11:42:50.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:43:22.258 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-12-17T11:43:22.683 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-17T11:43:22.683 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-gcd68252d7b4d"... 2023-12-17T11:43:22.683 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-12-17T11:43:22.726 INFO:teuthology.orchestra.run.smithi027.stdout:6.6.0-gcd68252d7b4d 2023-12-17T11:43:22.727 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-17T11:43:22.727 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-17T11:43:22.727 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-17T11:43:23.728 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-17T11:43:23.728 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-17T11:43:23.875 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2023-12-17T11:43:23.896 DEBUG:teuthology.parallel:result is None 2023-12-17T11:43:23.897 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-17T11:43:23.905 INFO:teuthology.task.internal:Creating test directory... 2023-12-17T11:43:23.905 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-17T11:43:23.909 DEBUG:teuthology.orchestra.run.smithi141:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-17T11:43:23.915 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-17T11:43:23.963 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-17T11:43:23.971 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-17T11:43:23.980 INFO:teuthology.task.internal:Creating archive directory... 2023-12-17T11:43:23.980 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-17T11:43:23.995 DEBUG:teuthology.orchestra.run.smithi141:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-17T11:43:24.000 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-17T11:43:24.062 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-17T11:43:24.070 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-17T11:43:24.070 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-17T11:43:24.075 DEBUG:teuthology.orchestra.run.smithi141:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-17T11:43:24.080 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-17T11:43:24.145 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-17T11:43:24.157 INFO:teuthology.orchestra.run.smithi141.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-17T11:43:24.161 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-17T11:43:24.173 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-17T11:43:24.182 INFO:teuthology.orchestra.run.smithi141.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-17T11:43:24.189 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-17T11:43:24.193 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-17T11:43:24.200 INFO:teuthology.task.internal:Configuring sudo... 2023-12-17T11:43:24.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-17T11:43:24.205 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-17T11:43:24.209 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-17T11:43:24.277 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-17T11:43:24.287 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-17T11:43:24.287 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-17T11:43:24.292 DEBUG:teuthology.orchestra.run.smithi141:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-17T11:43:24.296 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-17T11:43:24.338 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-17T11:43:24.440 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-17T11:43:24.512 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-17T11:43:24.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-17T11:43:24.655 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T11:43:24.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-17T11:43:24.726 DEBUG:teuthology.orchestra.run.smithi141:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-17T11:43:24.803 DEBUG:teuthology.orchestra.run.smithi141:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-17T11:43:24.884 DEBUG:teuthology.orchestra.run.smithi141:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-17T11:43:24.957 DEBUG:teuthology.orchestra.run.smithi141:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-17T11:43:25.025 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T11:43:25.025 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-17T11:43:25.093 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-17T11:43:25.168 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-17T11:43:25.254 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-17T11:43:25.325 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-17T11:43:25.408 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T11:43:25.408 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-17T11:43:25.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2023-12-17T11:43:25.481 DEBUG:teuthology.orchestra.run.smithi141:> sudo service rsyslog restart 2023-12-17T11:43:25.485 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2023-12-17T11:43:25.611 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-17T11:43:25.612 INFO:teuthology.orchestra.run.smithi141.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-17T11:43:25.642 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-17T11:43:26.096 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-17T11:43:26.104 INFO:teuthology.task.internal:Starting timer... 2023-12-17T11:43:26.104 INFO:teuthology.run_tasks:Running task pcp... 2023-12-17T11:43:26.114 INFO:teuthology.run_tasks:Running task selinux... 2023-12-17T11:43:26.124 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-12-17T11:43:26.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2023-12-17T11:43:26.255 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2023-12-17T11:43:26.258 DEBUG:teuthology.orchestra.run.smithi141:> sudo service auditd rotate 2023-12-17T11:43:26.388 INFO:teuthology.orchestra.run.smithi141.stdout:Rotating logs: 2023-12-17T11:43:26.392 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2023-12-17T11:43:26.557 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2023-12-17T11:43:26.560 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-17T11:43:26.561 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2023-12-17T11:43:26.610 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2023-12-17T11:43:26.610 DEBUG:teuthology.orchestra.run.smithi141:> /usr/sbin/getenforce 2023-12-17T11:43:26.662 INFO:teuthology.orchestra.run.smithi141.stdout:Permissive 2023-12-17T11:43:26.662 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2023-12-17T11:43:26.711 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2023-12-17T11:43:26.711 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi141.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2023-12-17T11:43:26.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-17T11:43:26.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T11:43:26.784 DEBUG:teuthology.orchestra.run.smithi141:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-17T11:43:26.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T11:43:26.858 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-17T11:43:26.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T11:43:26.927 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-17T11:43:26.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2023-12-17T11:43:26.999 DEBUG:teuthology.orchestra.run.smithi141:> sudo /usr/sbin/setenforce permissive 2023-12-17T11:43:27.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2023-12-17T11:43:27.158 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-17T11:43:27.168 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-17T11:43:27.173 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-17T11:43:27.303 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-17T11:43:27.321 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-17T11:43:27.322 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi141.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-17T11:51:46.511 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi141.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2023-12-17T11:51:46.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-17T11:51:46.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:51:46.603 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-12-17T11:51:46.713 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-17T11:51:46.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-12-17T11:51:46.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:51:46.797 DEBUG:teuthology.orchestra.run.smithi141:> true 2023-12-17T11:51:46.908 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-12-17T11:51:46.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-17T11:51:46.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-17T11:51:47.000 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-12-17T11:51:47.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-17T11:51:47.111 INFO:teuthology.run_tasks:Running task clock... 2023-12-17T11:51:47.121 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-17T11:51:47.121 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-17T11:51:47.121 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-17T11:51:47.126 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-17T11:51:47.126 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-17T11:51:47.131 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-17T11:51:47.131 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-17T11:51:47.199 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-17T11:51:47.201 INFO:teuthology.orchestra.run.smithi141.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-17T11:51:47.227 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-17T11:51:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-17T11:51:47.236 INFO:teuthology.orchestra.run.smithi141.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-17T11:51:47.261 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-17T11:51:47.327 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2023-12-17T11:51:47.327 INFO:teuthology.orchestra.run.smithi141.stderr:sudo: ntpd: command not found 2023-12-17T11:51:47.352 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2023-12-17T11:51:47.355 INFO:teuthology.orchestra.run.smithi141.stdout:506 Cannot talk to daemon 2023-12-17T11:51:47.357 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2023-12-17T11:51:47.381 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2023-12-17T11:51:47.389 INFO:teuthology.orchestra.run.smithi141.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-17T11:51:47.394 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-17T11:51:47.415 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-17T11:51:47.422 INFO:teuthology.orchestra.run.smithi141.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-17T11:51:47.429 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-17T11:51:47.449 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-17T11:51:47.518 INFO:teuthology.orchestra.run.smithi141.stderr:bash: ntpq: command not found 2023-12-17T11:51:47.524 INFO:teuthology.orchestra.run.smithi027.stderr:bash: ntpq: command not found 2023-12-17T11:51:47.525 INFO:teuthology.orchestra.run.smithi141.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-17T11:51:47.525 INFO:teuthology.orchestra.run.smithi141.stdout:=============================================================================== 2023-12-17T11:51:47.525 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.525 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.526 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.526 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.532 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-17T11:51:47.532 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-12-17T11:51:47.532 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.532 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.532 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.532 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.542 INFO:teuthology.orchestra.run.smithi192.stderr:bash: ntpq: command not found 2023-12-17T11:51:47.550 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-17T11:51:47.550 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-12-17T11:51:47.550 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.550 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.550 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.550 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T11:51:47.551 INFO:teuthology.run_tasks:Running task pexec... 2023-12-17T11:51:47.560 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-17T11:51:47.561 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi027.front.sepia.ceph.com 2023-12-17T11:51:47.561 DEBUG:teuthology.orchestra.run.smithi027:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-17T11:51:47.561 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi141.front.sepia.ceph.com 2023-12-17T11:51:47.562 DEBUG:teuthology.orchestra.run.smithi141:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-17T11:51:47.562 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi192.front.sepia.ceph.com 2023-12-17T11:51:47.562 DEBUG:teuthology.orchestra.run.smithi192:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-17T11:51:48.047 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-12-17T11:51:48.051 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-12-17T11:51:48.088 INFO:teuthology.orchestra.run.smithi027.stdout:Updating Subscription Management repositories. 2023-12-17T11:51:50.004 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 kB/s | 4.5 kB 00:00 2023-12-17T11:51:50.082 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2023-12-17T11:51:50.096 INFO:teuthology.orchestra.run.smithi027.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2023-12-17T11:51:50.489 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2023-12-17T11:51:50.620 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 kB/s | 4.1 kB 00:00 2023-12-17T11:51:50.806 INFO:teuthology.orchestra.run.smithi027.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2023-12-17T11:51:53.572 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-12-17T11:51:53.573 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-17T11:51:53.573 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2023-12-17T11:51:53.573 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-17T11:51:53.573 INFO:teuthology.orchestra.run.smithi192.stdout:Resetting modules: 2023-12-17T11:51:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools 2023-12-17T11:51:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:51:53.574 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-12-17T11:51:53.574 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-17T11:51:53.574 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:51:53.579 INFO:teuthology.orchestra.run.smithi141.stdout:Dependencies resolved. 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout:================================================================================ 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout: Package Architecture Version Repository Size 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout:================================================================================ 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout:Resetting modules: 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout: container-tools 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction Summary 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout:================================================================================ 2023-12-17T11:51:53.581 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:51:53.947 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout:Resetting modules: 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout: container-tools 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-12-17T11:51:53.949 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:52:04.442 INFO:teuthology.orchestra.run.smithi141.stdout:Complete! 2023-12-17T11:52:04.585 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-12-17T11:52:05.056 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:05.192 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:07.141 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 kB/s | 4.5 kB 00:00 2023-12-17T11:52:07.200 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2023-12-17T11:52:07.582 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 kB/s | 4.5 kB 00:00 2023-12-17T11:52:07.761 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 kB/s | 4.1 kB 00:00 2023-12-17T11:52:07.921 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-12-17T11:52:08.581 INFO:teuthology.orchestra.run.smithi027.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:10.649 INFO:teuthology.orchestra.run.smithi027.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2023-12-17T11:52:10.673 INFO:teuthology.orchestra.run.smithi141.stdout:Dependencies resolved. 2023-12-17T11:52:10.717 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-12-17T11:52:10.832 INFO:teuthology.orchestra.run.smithi141.stdout:========================================================================================================== 2023-12-17T11:52:10.832 INFO:teuthology.orchestra.run.smithi141.stdout: Package Arch Version Repository Size 2023-12-17T11:52:10.832 INFO:teuthology.orchestra.run.smithi141.stdout:========================================================================================================== 2023-12-17T11:52:10.832 INFO:teuthology.orchestra.run.smithi141.stdout:Installing group/module packages: 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout:Installing dependencies: 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-17T11:52:10.833 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout:Installing weak dependencies: 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout:Installing module profiles: 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout: container-tools/common 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout:Enabling module streams: 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout: container-tools rhel8 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction Summary 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout:========================================================================================================== 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout:Install 13 Packages 2023-12-17T11:52:10.834 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:52:10.835 INFO:teuthology.orchestra.run.smithi141.stdout:Total download size: 26 M 2023-12-17T11:52:10.835 INFO:teuthology.orchestra.run.smithi141.stdout:Installed size: 87 M 2023-12-17T11:52:10.835 INFO:teuthology.orchestra.run.smithi141.stdout:Downloading Packages: 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout:========================================================================================================== 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout:========================================================================================================== 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout:Installing group/module packages: 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-17T11:52:10.880 INFO:teuthology.orchestra.run.smithi192.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout:Installing module profiles: 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools/common 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout:Enabling module streams: 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools rhel8 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:52:10.881 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-12-17T11:52:10.882 INFO:teuthology.orchestra.run.smithi192.stdout:========================================================================================================== 2023-12-17T11:52:10.882 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2023-12-17T11:52:10.882 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:52:10.882 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 26 M 2023-12-17T11:52:10.882 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 87 M 2023-12-17T11:52:10.882 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-12-17T11:52:11.038 INFO:teuthology.orchestra.run.smithi141.stdout:(1/13): yajl-2.1.0-12.el8.x86_64.rpm 203 kB/s | 41 kB 00:00 2023-12-17T11:52:11.071 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 502 kB/s | 94 kB 00:00 2023-12-17T11:52:11.088 INFO:teuthology.orchestra.run.smithi141.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 98 kB/s | 25 kB 00:00 2023-12-17T11:52:11.096 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 117 kB/s | 25 kB 00:00 2023-12-17T11:52:11.121 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): yajl-2.1.0-12.el8.x86_64.rpm 172 kB/s | 41 kB 00:00 2023-12-17T11:52:11.139 INFO:teuthology.orchestra.run.smithi141.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 310 kB/s | 94 kB 00:00 2023-12-17T11:52:11.189 INFO:teuthology.orchestra.run.smithi141.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 6.7 MB/s | 1.0 MB 00:00 2023-12-17T11:52:11.264 INFO:teuthology.orchestra.run.smithi141.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 5.8 MB/s | 739 kB 00:00 2023-12-17T11:52:11.280 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 4.8 MB/s | 1.0 MB 00:00 2023-12-17T11:52:11.317 INFO:teuthology.orchestra.run.smithi027.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2023-12-17T11:52:11.322 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 3.6 MB/s | 739 kB 00:00 2023-12-17T11:52:11.323 INFO:teuthology.orchestra.run.smithi141.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 294 kB/s | 39 kB 00:00 2023-12-17T11:52:11.397 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 336 kB/s | 39 kB 00:00 2023-12-17T11:52:11.431 INFO:teuthology.orchestra.run.smithi141.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 1.4 MB/s | 239 kB 00:00 2023-12-17T11:52:11.472 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 1.6 MB/s | 239 kB 00:00 2023-12-17T11:52:11.548 INFO:teuthology.orchestra.run.smithi141.stdout:(8/13): buildah-1.31.3-1.module+el8.9.0+20326+3 19 MB/s | 8.8 MB 00:00 2023-12-17T11:52:11.573 INFO:teuthology.orchestra.run.smithi141.stdout:(9/13): python3-podman-4.6.0-1.module+el8.9.0+2 1.1 MB/s | 153 kB 00:00 2023-12-17T11:52:11.631 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): buildah-1.31.3-1.module+el8.9.0+20326+3 16 MB/s | 8.8 MB 00:00 2023-12-17T11:52:11.656 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): python3-podman-4.6.0-1.module+el8.9.0+2 836 kB/s | 153 kB 00:00 2023-12-17T11:52:11.665 INFO:teuthology.orchestra.run.smithi141.stdout:(10/13): netavark-1.7.0-1.module+el8.9.0+20326+ 11 MB/s | 3.7 MB 00:00 2023-12-17T11:52:11.740 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): netavark-1.7.0-1.module+el8.9.0+20326+ 11 MB/s | 3.7 MB 00:00 2023-12-17T11:52:11.749 INFO:teuthology.orchestra.run.smithi141.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 14 MB/s | 2.5 MB 00:00 2023-12-17T11:52:11.791 INFO:teuthology.orchestra.run.smithi141.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+20326+38 395 kB/s | 49 kB 00:00 2023-12-17T11:52:11.857 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+20326+38 424 kB/s | 49 kB 00:00 2023-12-17T11:52:11.907 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 10 MB/s | 2.5 MB 00:00 2023-12-17T11:52:11.941 INFO:teuthology.orchestra.run.smithi141.stdout:(13/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 21 MB/s | 8.1 MB 00:00 2023-12-17T11:52:11.942 INFO:teuthology.orchestra.run.smithi141.stdout:-------------------------------------------------------------------------------- 2023-12-17T11:52:11.942 INFO:teuthology.orchestra.run.smithi141.stdout:Total 23 MB/s | 26 MB 00:01 2023-12-17T11:52:12.040 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 20 MB/s | 8.1 MB 00:00 2023-12-17T11:52:12.041 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-12-17T11:52:12.041 INFO:teuthology.orchestra.run.smithi192.stdout:Total 22 MB/s | 26 MB 00:01 2023-12-17T11:52:12.401 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction check 2023-12-17T11:52:12.427 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction check succeeded. 2023-12-17T11:52:12.427 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction test 2023-12-17T11:52:12.504 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-12-17T11:52:12.530 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-12-17T11:52:12.530 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-12-17T11:52:13.056 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction test succeeded. 2023-12-17T11:52:13.065 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction 2023-12-17T11:52:13.200 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-12-17T11:52:13.211 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-12-17T11:52:13.714 INFO:teuthology.orchestra.run.smithi141.stdout: Preparing : 1/1 2023-12-17T11:52:13.876 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-12-17T11:52:13.969 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-17T11:52:14.127 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-17T11:52:14.362 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-12-17T11:52:14.534 INFO:teuthology.orchestra.run.smithi027.stdout:========================================================================================================== 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout:========================================================================================================== 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout:Installing group/module packages: 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-17T11:52:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout:Installing module profiles: 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: container-tools/common 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling module streams: 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: container-tools rhel8 2023-12-17T11:52:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:52:14.537 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-12-17T11:52:14.537 INFO:teuthology.orchestra.run.smithi027.stdout:========================================================================================================== 2023-12-17T11:52:14.537 INFO:teuthology.orchestra.run.smithi027.stdout:Install 13 Packages 2023-12-17T11:52:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:52:14.537 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 26 M 2023-12-17T11:52:14.537 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 87 M 2023-12-17T11:52:14.537 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-12-17T11:52:14.563 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-17T11:52:14.651 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-17T11:52:14.700 INFO:teuthology.orchestra.run.smithi027.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 583 kB/s | 94 kB 00:00 2023-12-17T11:52:14.725 INFO:teuthology.orchestra.run.smithi027.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 133 kB/s | 25 kB 00:00 2023-12-17T11:52:14.741 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-17T11:52:14.776 INFO:teuthology.orchestra.run.smithi027.stdout:(3/13): yajl-2.1.0-12.el8.x86_64.rpm 173 kB/s | 41 kB 00:00 2023-12-17T11:52:14.867 INFO:teuthology.orchestra.run.smithi027.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 6.0 MB/s | 1.0 MB 00:00 2023-12-17T11:52:14.872 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-17T11:52:14.934 INFO:teuthology.orchestra.run.smithi027.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 4.6 MB/s | 739 kB 00:00 2023-12-17T11:52:14.960 INFO:teuthology.orchestra.run.smithi027.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 428 kB/s | 39 kB 00:00 2023-12-17T11:52:14.965 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-17T11:52:15.040 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-17T11:52:15.149 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-17T11:52:15.210 INFO:teuthology.orchestra.run.smithi027.stdout:(7/13): buildah-1.31.3-1.module+el8.9.0+20326+3 18 MB/s | 8.8 MB 00:00 2023-12-17T11:52:15.243 INFO:teuthology.orchestra.run.smithi027.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+20326+38708 775 kB/s | 239 kB 00:00 2023-12-17T11:52:15.275 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-17T11:52:15.352 INFO:teuthology.orchestra.run.smithi027.stdout:(9/13): netavark-1.7.0-1.module+el8.9.0+20326+3 9.5 MB/s | 3.7 MB 00:00 2023-12-17T11:52:15.377 INFO:teuthology.orchestra.run.smithi027.stdout:(10/13): python3-podman-4.6.0-1.module+el8.9.0+ 919 kB/s | 153 kB 00:00 2023-12-17T11:52:15.401 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-17T11:52:15.519 INFO:teuthology.orchestra.run.smithi027.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+20326+38 349 kB/s | 49 kB 00:00 2023-12-17T11:52:15.545 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-17T11:52:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:(12/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 20 MB/s | 8.1 MB 00:00 2023-12-17T11:52:15.677 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-17T11:52:15.703 INFO:teuthology.orchestra.run.smithi027.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 7.2 MB/s | 2.5 MB 00:00 2023-12-17T11:52:15.704 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-12-17T11:52:15.704 INFO:teuthology.orchestra.run.smithi027.stdout:Total 22 MB/s | 26 MB 00:01 2023-12-17T11:52:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-17T11:52:16.181 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-12-17T11:52:16.209 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-12-17T11:52:16.209 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-12-17T11:52:16.630 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-17T11:52:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-17T11:52:16.897 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-12-17T11:52:16.898 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-17T11:52:16.907 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-12-17T11:52:17.045 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-17T11:52:17.300 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-17T11:52:17.447 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-17T11:52:17.600 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-12-17T11:52:17.854 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-17T11:52:18.303 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-17T11:52:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-17T11:52:18.452 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-17T11:52:18.557 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-17T11:52:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-17T11:52:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-17T11:52:18.763 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-17T11:52:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-17T11:52:18.875 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-17T11:52:19.075 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-17T11:52:19.378 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-17T11:52:19.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-17T11:52:19.455 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-17T11:52:19.455 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-17T11:52:19.455 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-17T11:52:19.455 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-17T11:52:19.551 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-17T11:52:19.552 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-17T11:52:19.552 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-17T11:52:19.552 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-17T11:52:19.604 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-17T11:52:20.594 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-17T11:52:20.822 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-17T11:52:21.248 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-17T11:52:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-17T11:52:22.530 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-17T11:52:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-17T11:52:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-17T11:52:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-17T11:52:30.678 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-17T11:52:30.678 INFO:teuthology.orchestra.run.smithi141.stdout:Installed products updated. 2023-12-17T11:52:31.305 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:52:31.305 INFO:teuthology.orchestra.run.smithi141.stdout:Installed: 2023-12-17T11:52:31.305 INFO:teuthology.orchestra.run.smithi141.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:31.305 INFO:teuthology.orchestra.run.smithi141.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:31.305 INFO:teuthology.orchestra.run.smithi141.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:31.305 INFO:teuthology.orchestra.run.smithi141.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:31.305 INFO:teuthology.orchestra.run.smithi141.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:31.305 INFO:teuthology.orchestra.run.smithi141.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:52:31.306 INFO:teuthology.orchestra.run.smithi141.stdout:Complete! 2023-12-17T11:52:31.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-17T11:52:31.471 INFO:teuthology.orchestra.run.smithi192.stdout:Installed products updated. 2023-12-17T11:52:31.553 DEBUG:teuthology.parallel:result is None 2023-12-17T11:52:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-12-17T11:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:52:32.102 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-12-17T11:52:32.387 DEBUG:teuthology.parallel:result is None 2023-12-17T11:52:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-17T11:52:34.490 INFO:teuthology.orchestra.run.smithi027.stdout:Installed products updated. 2023-12-17T11:52:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-17T11:52:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-17T11:52:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-17T11:52:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-17T11:52:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:52:35.154 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-12-17T11:52:35.428 DEBUG:teuthology.parallel:result is None 2023-12-17T11:52:35.428 INFO:teuthology.run_tasks:Running task install... 2023-12-17T11:52:35.436 DEBUG:teuthology.task.install:project ceph 2023-12-17T11:52:35.436 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d'}} 2023-12-17T11:52:35.436 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d'} 2023-12-17T11:52:35.436 INFO:teuthology.task.install:Using flavor: default 2023-12-17T11:52:35.442 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-17T11:52:35.442 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-17T11:52:35.442 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-17T11:52:35.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:52:35.445 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-17T11:52:35.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:52:35.446 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-17T11:52:35.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:52:51.102 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-17T11:52:51.102 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-17T11:52:51.228 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-17T11:52:51.228 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-17T11:52:51.234 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-17T11:52:51.234 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-17T11:52:51.535 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-17T11:52:51.535 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T11:52:51.536 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-17T11:52:51.538 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-17T11:52:51.540 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T11:52:51.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-17T11:52:51.619 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-17T11:52:51.620 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-17T11:52:51.621 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-17T11:52:51.621 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-17T11:52:51.668 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-17T11:52:51.668 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T11:52:51.668 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-17T11:52:51.731 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-17T11:52:51.736 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-17T11:52:51.745 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-17T11:52:51.745 DEBUG:teuthology.orchestra.run.smithi141:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-17T11:52:51.826 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-17T11:52:51.836 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-17T11:52:51.856 DEBUG:teuthology.orchestra.run.smithi141:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-17T11:52:51.893 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2023-12-17T11:52:51.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2023-12-17T11:52:51.905 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2023-12-17T11:52:51.908 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2023-12-17T11:52:51.951 DEBUG:teuthology.orchestra.run.smithi141:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-17T11:52:52.019 INFO:teuthology.orchestra.run.smithi141.stdout:check_obsoletes = 1 2023-12-17T11:52:52.022 DEBUG:teuthology.orchestra.run.smithi141:> sudo yum clean all 2023-12-17T11:52:52.346 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:52.401 INFO:teuthology.orchestra.run.smithi027.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:52.473 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:53.714 INFO:teuthology.orchestra.run.smithi192.stdout:52 files removed 2023-12-17T11:52:53.774 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-17T11:52:53.801 INFO:teuthology.orchestra.run.smithi141.stdout:52 files removed 2023-12-17T11:52:53.804 INFO:teuthology.orchestra.run.smithi027.stdout:52 files removed 2023-12-17T11:52:53.859 DEBUG:teuthology.orchestra.run.smithi141:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-17T11:52:53.862 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-17T11:52:54.225 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:54.313 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:54.357 INFO:teuthology.orchestra.run.smithi027.stdout:Updating Subscription Management repositories. 2023-12-17T11:52:56.718 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 84 kB/s | 81 kB 00:00 2023-12-17T11:52:56.771 INFO:teuthology.orchestra.run.smithi141.stdout:ceph packages for x86_64 92 kB/s | 81 kB 00:00 2023-12-17T11:52:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 104 kB/s | 81 kB 00:00 2023-12-17T11:52:57.486 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 13 kB/s | 9.5 kB 00:00 2023-12-17T11:52:57.638 INFO:teuthology.orchestra.run.smithi141.stdout:ceph noarch packages 11 kB/s | 9.5 kB 00:00 2023-12-17T11:52:57.749 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 12 kB/s | 9.5 kB 00:00 2023-12-17T11:52:58.345 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 1.8 kB/s | 1.5 kB 00:00 2023-12-17T11:52:58.461 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 2.2 kB/s | 1.5 kB 00:00 2023-12-17T11:52:58.505 INFO:teuthology.orchestra.run.smithi141.stdout:ceph source packages 1.8 kB/s | 1.5 kB 00:00 2023-12-17T11:52:58.562 INFO:teuthology.orchestra.run.smithi192.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-12-17T11:52:58.678 INFO:teuthology.orchestra.run.smithi027.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-12-17T11:52:58.697 INFO:teuthology.orchestra.run.smithi141.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-12-17T11:52:59.690 INFO:teuthology.orchestra.run.smithi141.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-12-17T11:52:59.865 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-12-17T11:53:03.573 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 16 MB 00:04 2023-12-17T11:53:03.840 INFO:teuthology.orchestra.run.smithi141.stdout:lab-extras 343 kB/s | 24 kB 00:00 2023-12-17T11:53:04.304 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 381 kB/s | 24 kB 00:00 2023-12-17T11:53:05.197 INFO:teuthology.orchestra.run.smithi027.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 11 MB/s | 8.8 MB 00:00 2023-12-17T11:53:05.350 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 64 MB 00:01 2023-12-17T11:53:07.704 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 404 kB/s | 24 kB 00:00 2023-12-17T11:53:08.397 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2023-12-17T11:53:08.491 INFO:teuthology.orchestra.run.smithi027.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 MB/s | 58 MB 00:01 2023-12-17T11:53:11.763 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 MB/s | 58 MB 00:01 2023-12-17T11:53:15.642 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 16 MB/s | 8.8 MB 00:00 2023-12-17T11:53:18.960 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 58 MB 00:01 2023-12-17T11:53:21.542 INFO:teuthology.orchestra.run.smithi027.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 MB/s | 64 MB 00:01 2023-12-17T11:53:23.565 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 64 MB 00:01 2023-12-17T11:53:35.052 INFO:teuthology.orchestra.run.smithi141.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-17T11:53:35.053 INFO:teuthology.orchestra.run.smithi141.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-17T11:53:35.072 INFO:teuthology.orchestra.run.smithi141.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-17T11:53:35.074 INFO:teuthology.orchestra.run.smithi141.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-17T11:53:35.075 INFO:teuthology.orchestra.run.smithi141.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-17T11:53:35.077 INFO:teuthology.orchestra.run.smithi141.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-17T11:53:35.080 INFO:teuthology.orchestra.run.smithi141.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-17T11:53:35.084 INFO:teuthology.orchestra.run.smithi141.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-17T11:53:35.085 INFO:teuthology.orchestra.run.smithi141.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-17T11:53:35.304 INFO:teuthology.orchestra.run.smithi141.stdout:Dependencies resolved. 2023-12-17T11:53:35.318 INFO:teuthology.orchestra.run.smithi141.stdout:=========================================================================================================================== 2023-12-17T11:53:35.318 INFO:teuthology.orchestra.run.smithi141.stdout: Package Arch Version Repository Size 2023-12-17T11:53:35.318 INFO:teuthology.orchestra.run.smithi141.stdout:=========================================================================================================================== 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout:Installing: 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-17T11:53:35.319 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-17T11:53:35.320 INFO:teuthology.orchestra.run.smithi141.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-17T11:53:35.321 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout:Upgrading: 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-17T11:53:35.322 INFO:teuthology.orchestra.run.smithi141.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-17T11:53:35.323 INFO:teuthology.orchestra.run.smithi141.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-17T11:53:35.323 INFO:teuthology.orchestra.run.smithi141.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-17T11:53:35.323 INFO:teuthology.orchestra.run.smithi141.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-17T11:53:35.323 INFO:teuthology.orchestra.run.smithi141.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-17T11:53:35.323 INFO:teuthology.orchestra.run.smithi141.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-17T11:53:35.323 INFO:teuthology.orchestra.run.smithi141.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-17T11:53:35.323 INFO:teuthology.orchestra.run.smithi141.stdout:Installing dependencies: 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-17T11:53:35.324 INFO:teuthology.orchestra.run.smithi141.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-17T11:53:35.325 INFO:teuthology.orchestra.run.smithi141.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-17T11:53:35.326 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-17T11:53:35.327 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-17T11:53:35.328 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-17T11:53:35.329 INFO:teuthology.orchestra.run.smithi141.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout:Enabling module streams: 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: postgresql 10 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction Summary 2023-12-17T11:53:35.330 INFO:teuthology.orchestra.run.smithi141.stdout:=========================================================================================================================== 2023-12-17T11:53:35.331 INFO:teuthology.orchestra.run.smithi141.stdout:Install 130 Packages 2023-12-17T11:53:35.331 INFO:teuthology.orchestra.run.smithi141.stdout:Upgrade 16 Packages 2023-12-17T11:53:35.331 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:53:35.331 INFO:teuthology.orchestra.run.smithi141.stdout:Total download size: 218 M 2023-12-17T11:53:35.331 INFO:teuthology.orchestra.run.smithi141.stdout:Downloading Packages: 2023-12-17T11:53:36.093 INFO:teuthology.orchestra.run.smithi141.stdout:(1/146): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 16 kB/s | 6.4 kB 00:00 2023-12-17T11:53:38.227 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-17T11:53:38.228 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-17T11:53:38.248 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-17T11:53:38.250 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-17T11:53:38.251 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-17T11:53:38.253 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-17T11:53:38.256 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-17T11:53:38.260 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-17T11:53:38.262 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-17T11:53:38.510 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-12-17T11:53:38.526 INFO:teuthology.orchestra.run.smithi027.stdout:=========================================================================================================================== 2023-12-17T11:53:38.526 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2023-12-17T11:53:38.526 INFO:teuthology.orchestra.run.smithi027.stdout:=========================================================================================================================== 2023-12-17T11:53:38.526 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2023-12-17T11:53:38.526 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-17T11:53:38.527 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-17T11:53:38.528 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-17T11:53:38.529 INFO:teuthology.orchestra.run.smithi027.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-17T11:53:38.530 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-17T11:53:38.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-17T11:53:38.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-17T11:53:38.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-17T11:53:38.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-17T11:53:38.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-17T11:53:38.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-17T11:53:38.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-17T11:53:38.531 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-17T11:53:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-17T11:53:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-17T11:53:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-17T11:53:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-17T11:53:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-17T11:53:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-17T11:53:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-17T11:53:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-17T11:53:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-17T11:53:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-17T11:53:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-17T11:53:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling module streams: 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql 10 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:=========================================================================================================================== 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:Install 130 Packages 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 16 Packages 2023-12-17T11:53:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:53:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 218 M 2023-12-17T11:53:38.541 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-12-17T11:53:38.551 INFO:teuthology.orchestra.run.smithi141.stdout:(2/146): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 374 kB/s | 919 kB 00:02 2023-12-17T11:53:38.695 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-17T11:53:38.696 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-17T11:53:38.715 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-17T11:53:38.717 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-17T11:53:38.718 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-17T11:53:38.720 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-17T11:53:38.723 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-17T11:53:38.727 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-17T11:53:38.729 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-17T11:53:38.949 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-12-17T11:53:38.959 INFO:teuthology.orchestra.run.smithi141.stdout:(3/146): ceph-immutable-object-cache-19.0.0-119 419 kB/s | 171 kB 00:00 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-17T11:53:38.964 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-17T11:53:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-17T11:53:38.966 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-17T11:53:38.967 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-17T11:53:38.968 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-17T11:53:38.969 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-17T11:53:38.970 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-17T11:53:38.971 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-17T11:53:38.972 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-17T11:53:38.973 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-17T11:53:38.974 INFO:teuthology.orchestra.run.smithi192.stdout:Enabling module streams: 2023-12-17T11:53:38.975 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql 10 2023-12-17T11:53:38.975 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:53:38.975 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-12-17T11:53:38.975 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-12-17T11:53:38.975 INFO:teuthology.orchestra.run.smithi192.stdout:Install 130 Packages 2023-12-17T11:53:38.975 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 16 Packages 2023-12-17T11:53:38.975 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:53:38.976 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 218 M 2023-12-17T11:53:38.976 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-12-17T11:53:39.117 INFO:teuthology.orchestra.run.smithi027.stdout:(1/146): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 17 kB/s | 6.4 kB 00:00 2023-12-17T11:53:39.521 INFO:teuthology.orchestra.run.smithi192.stdout:(1/146): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 14 kB/s | 6.4 kB 00:00 2023-12-17T11:53:40.076 INFO:teuthology.orchestra.run.smithi027.stdout:(2/146): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 960 kB/s | 919 kB 00:00 2023-12-17T11:53:40.259 INFO:teuthology.orchestra.run.smithi027.stdout:(3/146): ceph-immutable-object-cache-19.0.0-119 932 kB/s | 171 kB 00:00 2023-12-17T11:53:41.343 INFO:teuthology.orchestra.run.smithi027.stdout:(4/146): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 2.1 MB/s | 2.3 MB 00:01 2023-12-17T11:53:41.499 INFO:teuthology.orchestra.run.smithi192.stdout:(2/146): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 465 kB/s | 919 kB 00:01 2023-12-17T11:53:41.885 INFO:teuthology.orchestra.run.smithi027.stdout:(5/146): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 3.0 MB/s | 1.6 MB 00:00 2023-12-17T11:53:41.916 INFO:teuthology.orchestra.run.smithi192.stdout:(3/146): ceph-immutable-object-cache-19.0.0-119 410 kB/s | 171 kB 00:00 2023-12-17T11:53:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:(6/146): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 2.9 MB/s | 4.6 MB 00:01 2023-12-17T11:53:44.041 INFO:teuthology.orchestra.run.smithi192.stdout:(4/146): ceph-base-19.0.0-119.ga62fda73.el8.x86 1.1 MB/s | 5.4 MB 00:04 2023-12-17T11:53:44.184 INFO:teuthology.orchestra.run.smithi141.stdout:(4/146): ceph-base-19.0.0-119.ga62fda73.el8.x86 647 kB/s | 5.4 MB 00:08 2023-12-17T11:53:45.432 INFO:teuthology.orchestra.run.smithi192.stdout:(5/146): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 1.2 MB/s | 1.6 MB 00:01 2023-12-17T11:53:45.599 INFO:teuthology.orchestra.run.smithi192.stdout:(6/146): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 642 kB/s | 2.3 MB 00:03 2023-12-17T11:53:46.092 INFO:teuthology.orchestra.run.smithi141.stdout:(5/146): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 873 kB/s | 1.6 MB 00:01 2023-12-17T11:53:46.318 INFO:teuthology.orchestra.run.smithi141.stdout:(6/146): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 322 kB/s | 2.3 MB 00:07 2023-12-17T11:53:47.352 INFO:teuthology.orchestra.run.smithi027.stdout:(7/146): ceph-base-19.0.0-119.ga62fda73.el8.x86 638 kB/s | 5.4 MB 00:08 2023-12-17T11:53:52.083 INFO:teuthology.orchestra.run.smithi192.stdout:(7/146): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 715 kB/s | 4.6 MB 00:06 2023-12-17T11:53:52.709 INFO:teuthology.orchestra.run.smithi141.stdout:(7/146): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 719 kB/s | 4.6 MB 00:06 2023-12-17T11:53:53.726 INFO:teuthology.orchestra.run.smithi027.stdout:(8/146): ceph-osd-19.0.0-119.ga62fda73.el8.x86_ 1.8 MB/s | 18 MB 00:10 2023-12-17T11:53:53.835 INFO:teuthology.orchestra.run.smithi027.stdout:(9/146): ceph-selinux-19.0.0-119.ga62fda73.el8. 225 kB/s | 24 kB 00:00 2023-12-17T11:53:59.291 INFO:teuthology.orchestra.run.smithi192.stdout:(8/146): ceph-common-19.0.0-119.ga62fda73.el8.x 1.1 MB/s | 23 MB 00:20 2023-12-17T11:53:59.416 INFO:teuthology.orchestra.run.smithi192.stdout:(9/146): ceph-selinux-19.0.0-119.ga62fda73.el8. 195 kB/s | 24 kB 00:00 2023-12-17T11:54:05.499 INFO:teuthology.orchestra.run.smithi141.stdout:(8/146): ceph-radosgw-19.0.0-119.ga62fda73.el8. 1.1 MB/s | 14 MB 00:12 2023-12-17T11:54:05.591 INFO:teuthology.orchestra.run.smithi141.stdout:(9/146): ceph-selinux-19.0.0-119.ga62fda73.el8. 266 kB/s | 24 kB 00:00 2023-12-17T11:54:07.375 INFO:teuthology.orchestra.run.smithi141.stdout:(10/146): ceph-common-19.0.0-119.ga62fda73.el8. 728 kB/s | 23 MB 00:31 2023-12-17T11:54:07.908 INFO:teuthology.orchestra.run.smithi141.stdout:(11/146): cephfs-mirror-19.0.0-119.ga62fda73.el 493 kB/s | 262 kB 00:00 2023-12-17T11:54:08.008 INFO:teuthology.orchestra.run.smithi141.stdout:(12/146): libcephfs-devel-19.0.0-119.ga62fda73. 311 kB/s | 31 kB 00:00 2023-12-17T11:54:08.376 INFO:teuthology.orchestra.run.smithi027.stdout:(10/146): ceph-radosgw-19.0.0-119.ga62fda73.el8 675 kB/s | 14 MB 00:21 2023-12-17T11:54:08.659 INFO:teuthology.orchestra.run.smithi027.stdout:(11/146): cephfs-mirror-19.0.0-119.ga62fda73.el 928 kB/s | 262 kB 00:00 2023-12-17T11:54:08.776 INFO:teuthology.orchestra.run.smithi027.stdout:(12/146): libcephfs-devel-19.0.0-119.ga62fda73. 266 kB/s | 31 kB 00:00 2023-12-17T11:54:09.184 INFO:teuthology.orchestra.run.smithi141.stdout:(13/146): libcephfs2-19.0.0-119.ga62fda73.el8.x 714 kB/s | 838 kB 00:01 2023-12-17T11:54:09.426 INFO:teuthology.orchestra.run.smithi141.stdout:(14/146): libcephsqlite-19.0.0-119.ga62fda73.el 757 kB/s | 183 kB 00:00 2023-12-17T11:54:09.452 INFO:teuthology.orchestra.run.smithi027.stdout:(13/146): libcephfs2-19.0.0-119.ga62fda73.el8.x 1.2 MB/s | 838 kB 00:00 2023-12-17T11:54:09.675 INFO:teuthology.orchestra.run.smithi141.stdout:(15/146): librados-devel-19.0.0-119.ga62fda73.e 557 kB/s | 139 kB 00:00 2023-12-17T11:54:09.802 INFO:teuthology.orchestra.run.smithi027.stdout:(14/146): libcephsqlite-19.0.0-119.ga62fda73.el 522 kB/s | 183 kB 00:00 2023-12-17T11:54:09.936 INFO:teuthology.orchestra.run.smithi027.stdout:(15/146): librados-devel-19.0.0-119.ga62fda73.e 1.0 MB/s | 139 kB 00:00 2023-12-17T11:54:10.328 INFO:teuthology.orchestra.run.smithi027.stdout:(16/146): libradosstriper1-19.0.0-119.ga62fda73 1.3 MB/s | 517 kB 00:00 2023-12-17T11:54:10.459 INFO:teuthology.orchestra.run.smithi141.stdout:(16/146): libradosstriper1-19.0.0-119.ga62fda73 660 kB/s | 517 kB 00:00 2023-12-17T11:54:11.773 INFO:teuthology.orchestra.run.smithi192.stdout:(10/146): ceph-osd-19.0.0-119.ga62fda73.el8.x86 705 kB/s | 18 MB 00:26 2023-12-17T11:54:11.993 INFO:teuthology.orchestra.run.smithi141.stdout:(17/146): ceph-osd-19.0.0-119.ga62fda73.el8.x86 718 kB/s | 18 MB 00:25 2023-12-17T11:54:12.256 INFO:teuthology.orchestra.run.smithi192.stdout:(11/146): cephfs-mirror-19.0.0-119.ga62fda73.el 544 kB/s | 262 kB 00:00 2023-12-17T11:54:12.365 INFO:teuthology.orchestra.run.smithi192.stdout:(12/146): libcephfs-devel-19.0.0-119.ga62fda73. 287 kB/s | 31 kB 00:00 2023-12-17T11:54:12.698 INFO:teuthology.orchestra.run.smithi141.stdout:(18/146): python3-ceph-argparse-19.0.0-119.ga62 63 kB/s | 45 kB 00:00 2023-12-17T11:54:12.732 INFO:teuthology.orchestra.run.smithi192.stdout:(13/146): ceph-radosgw-19.0.0-119.ga62fda73.el8 687 kB/s | 14 MB 00:20 2023-12-17T11:54:12.910 INFO:teuthology.orchestra.run.smithi141.stdout:(19/146): python3-ceph-common-19.0.0-119.ga62fd 609 kB/s | 129 kB 00:00 2023-12-17T11:54:13.202 INFO:teuthology.orchestra.run.smithi141.stdout:(20/146): python3-cephfs-19.0.0-119.ga62fda73.e 676 kB/s | 197 kB 00:00 2023-12-17T11:54:13.207 INFO:teuthology.orchestra.run.smithi192.stdout:(14/146): libcephsqlite-19.0.0-119.ga62fda73.el 385 kB/s | 183 kB 00:00 2023-12-17T11:54:13.616 INFO:teuthology.orchestra.run.smithi192.stdout:(15/146): librados-devel-19.0.0-119.ga62fda73.e 340 kB/s | 139 kB 00:00 2023-12-17T11:54:13.702 INFO:teuthology.orchestra.run.smithi141.stdout:(21/146): python3-rados-19.0.0-119.ga62fda73.el 743 kB/s | 371 kB 00:00 2023-12-17T11:54:13.799 INFO:teuthology.orchestra.run.smithi192.stdout:(16/146): libcephfs2-19.0.0-119.ga62fda73.el8.x 585 kB/s | 838 kB 00:01 2023-12-17T11:54:14.094 INFO:teuthology.orchestra.run.smithi141.stdout:(22/146): python3-rbd-19.0.0-119.ga62fda73.el8. 930 kB/s | 364 kB 00:00 2023-12-17T11:54:14.277 INFO:teuthology.orchestra.run.smithi141.stdout:(23/146): python3-rgw-19.0.0-119.ga62fda73.el8. 583 kB/s | 107 kB 00:00 2023-12-17T11:54:14.394 INFO:teuthology.orchestra.run.smithi141.stdout:(24/146): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 802 kB/s | 93 kB 00:00 2023-12-17T11:54:14.633 INFO:teuthology.orchestra.run.smithi192.stdout:(17/146): libradosstriper1-19.0.0-119.ga62fda73 509 kB/s | 517 kB 00:01 2023-12-17T11:54:14.750 INFO:teuthology.orchestra.run.smithi192.stdout:(18/146): python3-ceph-argparse-19.0.0-119.ga62 384 kB/s | 45 kB 00:00 2023-12-17T11:54:15.042 INFO:teuthology.orchestra.run.smithi192.stdout:(19/146): python3-ceph-common-19.0.0-119.ga62fd 441 kB/s | 129 kB 00:00 2023-12-17T11:54:15.425 INFO:teuthology.orchestra.run.smithi192.stdout:(20/146): python3-cephfs-19.0.0-119.ga62fda73.e 514 kB/s | 197 kB 00:00 2023-12-17T11:54:16.044 INFO:teuthology.orchestra.run.smithi027.stdout:(17/146): ceph-common-19.0.0-119.ga62fda73.el8. 619 kB/s | 23 MB 00:37 2023-12-17T11:54:16.142 INFO:teuthology.orchestra.run.smithi192.stdout:(21/146): python3-rados-19.0.0-119.ga62fda73.el 518 kB/s | 371 kB 00:00 2023-12-17T11:54:16.261 INFO:teuthology.orchestra.run.smithi027.stdout:(18/146): python3-ceph-argparse-19.0.0-119.ga62 207 kB/s | 45 kB 00:00 2023-12-17T11:54:16.478 INFO:teuthology.orchestra.run.smithi027.stdout:(19/146): librgw2-19.0.0-119.ga62fda73.el8.x86_ 1.1 MB/s | 6.9 MB 00:06 2023-12-17T11:54:16.811 INFO:teuthology.orchestra.run.smithi027.stdout:(20/146): python3-cephfs-19.0.0-119.ga62fda73.e 592 kB/s | 197 kB 00:00 2023-12-17T11:54:16.937 INFO:teuthology.orchestra.run.smithi027.stdout:(21/146): python3-ceph-common-19.0.0-119.ga62fd 190 kB/s | 129 kB 00:00 2023-12-17T11:54:17.009 INFO:teuthology.orchestra.run.smithi192.stdout:(22/146): python3-rbd-19.0.0-119.ga62fda73.el8. 420 kB/s | 364 kB 00:00 2023-12-17T11:54:17.317 INFO:teuthology.orchestra.run.smithi192.stdout:(23/146): python3-rgw-19.0.0-119.ga62fda73.el8. 346 kB/s | 107 kB 00:00 2023-12-17T11:54:17.379 INFO:teuthology.orchestra.run.smithi027.stdout:(22/146): python3-rados-19.0.0-119.ga62fda73.el 655 kB/s | 371 kB 00:00 2023-12-17T11:54:17.535 INFO:teuthology.orchestra.run.smithi192.stdout:(24/146): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 431 kB/s | 93 kB 00:00 2023-12-17T11:54:17.587 INFO:teuthology.orchestra.run.smithi027.stdout:(23/146): python3-rgw-19.0.0-119.ga62fda73.el8. 512 kB/s | 107 kB 00:00 2023-12-17T11:54:17.804 INFO:teuthology.orchestra.run.smithi027.stdout:(24/146): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 432 kB/s | 93 kB 00:00 2023-12-17T11:54:17.988 INFO:teuthology.orchestra.run.smithi027.stdout:(25/146): python3-rbd-19.0.0-119.ga62fda73.el8. 347 kB/s | 364 kB 00:01 2023-12-17T11:54:18.471 INFO:teuthology.orchestra.run.smithi027.stdout:(26/146): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 375 kB/s | 181 kB 00:00 2023-12-17T11:54:18.589 INFO:teuthology.orchestra.run.smithi027.stdout:(27/146): ceph-grafana-dashboards-19.0.0-119.ga 240 kB/s | 28 kB 00:00 2023-12-17T11:54:18.810 INFO:teuthology.orchestra.run.smithi141.stdout:(25/146): rbd-mirror-19.0.0-119.ga62fda73.el8.x 832 kB/s | 3.6 MB 00:04 2023-12-17T11:54:19.005 INFO:teuthology.orchestra.run.smithi027.stdout:(28/146): ceph-mgr-cephadm-19.0.0-119.ga62fda73 318 kB/s | 133 kB 00:00 2023-12-17T11:54:19.210 INFO:teuthology.orchestra.run.smithi141.stdout:(26/146): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 453 kB/s | 181 kB 00:00 2023-12-17T11:54:19.327 INFO:teuthology.orchestra.run.smithi141.stdout:(27/146): ceph-grafana-dashboards-19.0.0-119.ga 241 kB/s | 28 kB 00:00 2023-12-17T11:54:19.628 INFO:teuthology.orchestra.run.smithi141.stdout:(28/146): ceph-mgr-cephadm-19.0.0-119.ga62fda73 442 kB/s | 133 kB 00:00 2023-12-17T11:54:21.503 INFO:teuthology.orchestra.run.smithi141.stdout:(29/146): librgw2-19.0.0-119.ga62fda73.el8.x86_ 641 kB/s | 6.9 MB 00:11 2023-12-17T11:54:22.356 INFO:teuthology.orchestra.run.smithi027.stdout:(29/146): rbd-mirror-19.0.0-119.ga62fda73.el8.x 807 kB/s | 3.6 MB 00:04 2023-12-17T11:54:22.406 INFO:teuthology.orchestra.run.smithi027.stdout:(30/146): ceph-mgr-dashboard-19.0.0-119.ga62fda 531 kB/s | 1.8 MB 00:03 2023-12-17T11:54:22.501 INFO:teuthology.orchestra.run.smithi192.stdout:(25/146): rbd-mirror-19.0.0-119.ga62fda73.el8.x 740 kB/s | 3.6 MB 00:04 2023-12-17T11:54:22.792 INFO:teuthology.orchestra.run.smithi192.stdout:(26/146): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 622 kB/s | 181 kB 00:00 2023-12-17T11:54:22.872 INFO:teuthology.orchestra.run.smithi027.stdout:(31/146): ceph-mgr-modules-core-19.0.0-119.ga62 521 kB/s | 243 kB 00:00 2023-12-17T11:54:22.943 INFO:teuthology.orchestra.run.smithi192.stdout:(27/146): ceph-grafana-dashboards-19.0.0-119.ga 187 kB/s | 28 kB 00:00 2023-12-17T11:54:23.023 INFO:teuthology.orchestra.run.smithi027.stdout:(32/146): ceph-mgr-rook-19.0.0-119.ga62fda73.el 334 kB/s | 50 kB 00:00 2023-12-17T11:54:23.140 INFO:teuthology.orchestra.run.smithi027.stdout:(33/146): ceph-prometheus-alerts-19.0.0-119.ga6 129 kB/s | 15 kB 00:00 2023-12-17T11:54:23.160 INFO:teuthology.orchestra.run.smithi192.stdout:(28/146): ceph-mgr-cephadm-19.0.0-119.ga62fda73 612 kB/s | 133 kB 00:00 2023-12-17T11:54:23.211 INFO:teuthology.orchestra.run.smithi141.stdout:(30/146): ceph-mgr-dashboard-19.0.0-119.ga62fda 504 kB/s | 1.8 MB 00:03 2023-12-17T11:54:23.656 INFO:teuthology.orchestra.run.smithi027.stdout:(34/146): ceph-volume-19.0.0-119.ga62fda73.el8. 507 kB/s | 262 kB 00:00 2023-12-17T11:54:23.703 INFO:teuthology.orchestra.run.smithi141.stdout:(31/146): ceph-mgr-modules-core-19.0.0-119.ga62 495 kB/s | 243 kB 00:00 2023-12-17T11:54:23.828 INFO:teuthology.orchestra.run.smithi141.stdout:(32/146): ceph-mgr-rook-19.0.0-119.ga62fda73.el 401 kB/s | 50 kB 00:00 2023-12-17T11:54:23.954 INFO:teuthology.orchestra.run.smithi141.stdout:(33/146): ceph-prometheus-alerts-19.0.0-119.ga6 120 kB/s | 15 kB 00:00 2023-12-17T11:54:24.415 INFO:teuthology.orchestra.run.smithi027.stdout:(35/146): cephadm-19.0.0-119.ga62fda73.el8.noar 548 kB/s | 415 kB 00:00 2023-12-17T11:54:24.454 INFO:teuthology.orchestra.run.smithi141.stdout:(34/146): ceph-volume-19.0.0-119.ga62fda73.el8. 524 kB/s | 262 kB 00:00 2023-12-17T11:54:24.524 INFO:teuthology.orchestra.run.smithi027.stdout:(36/146): cephfs-top-19.0.0-119.ga62fda73.el8.n 167 kB/s | 18 kB 00:00 2023-12-17T11:54:24.691 INFO:teuthology.orchestra.run.smithi027.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.8 MB/s | 306 kB 00:00 2023-12-17T11:54:24.733 INFO:teuthology.orchestra.run.smithi027.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 1.4 MB/s | 59 kB 00:00 2023-12-17T11:54:24.783 INFO:teuthology.orchestra.run.smithi027.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-12-17T11:54:25.010 INFO:teuthology.orchestra.run.smithi192.stdout:(29/146): ceph-mgr-dashboard-19.0.0-119.ga62fda 976 kB/s | 1.8 MB 00:01 2023-12-17T11:54:25.025 INFO:teuthology.orchestra.run.smithi027.stdout:(40/146): luarocks-3.5.0-1.el8.noarch.rpm 692 kB/s | 167 kB 00:00 2023-12-17T11:54:25.129 INFO:teuthology.orchestra.run.smithi141.stdout:(35/146): cephadm-19.0.0-119.ga62fda73.el8.noar 616 kB/s | 415 kB 00:00 2023-12-17T11:54:25.234 INFO:teuthology.orchestra.run.smithi027.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2023-12-17T11:54:25.254 INFO:teuthology.orchestra.run.smithi141.stdout:(36/146): cephfs-top-19.0.0-119.ga62fda73.el8.n 145 kB/s | 18 kB 00:00 2023-12-17T11:54:25.276 INFO:teuthology.orchestra.run.smithi027.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-12-17T11:54:25.326 INFO:teuthology.orchestra.run.smithi027.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-12-17T11:54:25.376 INFO:teuthology.orchestra.run.smithi027.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 662 kB/s | 33 kB 00:00 2023-12-17T11:54:25.427 INFO:teuthology.orchestra.run.smithi027.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2023-12-17T11:54:25.477 INFO:teuthology.orchestra.run.smithi027.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-12-17T11:54:25.527 INFO:teuthology.orchestra.run.smithi027.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2023-12-17T11:54:25.586 INFO:teuthology.orchestra.run.smithi027.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-12-17T11:54:25.645 INFO:teuthology.orchestra.run.smithi027.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-12-17T11:54:25.671 INFO:teuthology.orchestra.run.smithi141.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 734 kB/s | 306 kB 00:00 2023-12-17T11:54:25.687 INFO:teuthology.orchestra.run.smithi027.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 425 kB/s | 18 kB 00:00 2023-12-17T11:54:25.738 INFO:teuthology.orchestra.run.smithi141.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 885 kB/s | 59 kB 00:00 2023-12-17T11:54:25.762 INFO:teuthology.orchestra.run.smithi027.stdout:(51/146): python3-kubernetes-11.0.0-6.el8.noarc 16 MB/s | 1.2 MB 00:00 2023-12-17T11:54:25.796 INFO:teuthology.orchestra.run.smithi027.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-12-17T11:54:25.805 INFO:teuthology.orchestra.run.smithi141.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-12-17T11:54:25.810 INFO:teuthology.orchestra.run.smithi192.stdout:(30/146): librgw2-19.0.0-119.ga62fda73.el8.x86_ 589 kB/s | 6.9 MB 00:12 2023-12-17T11:54:25.838 INFO:teuthology.orchestra.run.smithi027.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-12-17T11:54:25.880 INFO:teuthology.orchestra.run.smithi027.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-12-17T11:54:25.938 INFO:teuthology.orchestra.run.smithi027.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-12-17T11:54:25.955 INFO:teuthology.orchestra.run.smithi141.stdout:(40/146): luarocks-3.5.0-1.el8.noarch.rpm 1.1 MB/s | 167 kB 00:00 2023-12-17T11:54:25.972 INFO:teuthology.orchestra.run.smithi027.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 473 kB/s | 16 kB 00:00 2023-12-17T11:54:26.006 INFO:teuthology.orchestra.run.smithi027.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-12-17T11:54:26.056 INFO:teuthology.orchestra.run.smithi027.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 3.9 MB/s | 196 kB 00:00 2023-12-17T11:54:26.106 INFO:teuthology.orchestra.run.smithi027.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-12-17T11:54:26.157 INFO:teuthology.orchestra.run.smithi027.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 376 kB/s | 19 kB 00:00 2023-12-17T11:54:26.199 INFO:teuthology.orchestra.run.smithi027.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 576 kB/s | 24 kB 00:00 2023-12-17T11:54:26.274 INFO:teuthology.orchestra.run.smithi027.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 391 kB/s | 29 kB 00:00 2023-12-17T11:54:26.297 INFO:teuthology.orchestra.run.smithi141.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-12-17T11:54:26.316 INFO:teuthology.orchestra.run.smithi027.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 664 kB/s | 27 kB 00:00 2023-12-17T11:54:26.356 INFO:teuthology.orchestra.run.smithi141.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 749 kB/s | 44 kB 00:00 2023-12-17T11:54:26.375 INFO:teuthology.orchestra.run.smithi027.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2023-12-17T11:54:26.385 INFO:teuthology.orchestra.run.smithi192.stdout:(31/146): ceph-mgr-modules-core-19.0.0-119.ga62 423 kB/s | 243 kB 00:00 2023-12-17T11:54:26.425 INFO:teuthology.orchestra.run.smithi027.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-12-17T11:54:26.476 INFO:teuthology.orchestra.run.smithi027.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-12-17T11:54:26.506 INFO:teuthology.orchestra.run.smithi141.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2023-12-17T11:54:26.517 INFO:teuthology.orchestra.run.smithi027.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-12-17T11:54:26.559 INFO:teuthology.orchestra.run.smithi027.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 554 kB/s | 23 kB 00:00 2023-12-17T11:54:26.573 INFO:teuthology.orchestra.run.smithi141.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 496 kB/s | 33 kB 00:00 2023-12-17T11:54:26.586 INFO:teuthology.orchestra.run.smithi192.stdout:(32/146): ceph-mgr-rook-19.0.0-119.ga62fda73.el 251 kB/s | 50 kB 00:00 2023-12-17T11:54:26.635 INFO:teuthology.orchestra.run.smithi027.stdout:(69/146): thrift-0.13.0-2.el8.x86_64.rpm 23 MB/s | 1.7 MB 00:00 2023-12-17T11:54:26.657 INFO:teuthology.orchestra.run.smithi141.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 196 kB/s | 16 kB 00:00 2023-12-17T11:54:26.660 INFO:teuthology.orchestra.run.smithi027.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-12-17T11:54:26.711 INFO:teuthology.orchestra.run.smithi192.stdout:(33/146): ceph-prometheus-alerts-19.0.0-119.ga6 120 kB/s | 15 kB 00:00 2023-12-17T11:54:26.807 INFO:teuthology.orchestra.run.smithi141.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-12-17T11:54:26.819 INFO:teuthology.orchestra.run.smithi027.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 126 kB/s | 20 kB 00:00 2023-12-17T11:54:26.986 INFO:teuthology.orchestra.run.smithi027.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 264 kB/s | 44 kB 00:00 2023-12-17T11:54:27.083 INFO:teuthology.orchestra.run.smithi141.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 1.4 MB/s | 384 kB 00:00 2023-12-17T11:54:27.136 INFO:teuthology.orchestra.run.smithi027.stdout:(73/146): python3-pyasn1-modules-0.3.7-6.el8.no 734 kB/s | 110 kB 00:00 2023-12-17T11:54:27.200 INFO:teuthology.orchestra.run.smithi141.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 698 kB/s | 81 kB 00:00 2023-12-17T11:54:27.266 INFO:teuthology.orchestra.run.smithi141.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-12-17T11:54:27.295 INFO:teuthology.orchestra.run.smithi192.stdout:(34/146): ceph-volume-19.0.0-119.ga62fda73.el8. 448 kB/s | 262 kB 00:00 2023-12-17T11:54:27.320 INFO:teuthology.orchestra.run.smithi027.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 145 kB/s | 27 kB 00:00 2023-12-17T11:54:27.342 INFO:teuthology.orchestra.run.smithi141.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 235 kB/s | 18 kB 00:00 2023-12-17T11:54:27.412 INFO:teuthology.orchestra.run.smithi027.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.3 MB/s | 126 kB 00:00 2023-12-17T11:54:27.695 INFO:teuthology.orchestra.run.smithi027.stdout:(76/146): python3-werkzeug-0.12.2-4.el8.noarch. 1.6 MB/s | 457 kB 00:00 2023-12-17T11:54:27.837 INFO:teuthology.orchestra.run.smithi027.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 4.8 MB/s | 688 kB 00:00 2023-12-17T11:54:27.946 INFO:teuthology.orchestra.run.smithi027.stdout:(78/146): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-12-17T11:54:28.061 INFO:teuthology.orchestra.run.smithi192.stdout:(35/146): cephadm-19.0.0-119.ga62fda73.el8.noar 542 kB/s | 415 kB 00:00 2023-12-17T11:54:28.109 INFO:teuthology.orchestra.run.smithi141.stdout:(51/146): python3-kubernetes-11.0.0-6.el8.noarc 1.5 MB/s | 1.2 MB 00:00 2023-12-17T11:54:28.170 INFO:teuthology.orchestra.run.smithi192.stdout:(36/146): cephfs-top-19.0.0-119.ga62fda73.el8.n 167 kB/s | 18 kB 00:00 2023-12-17T11:54:28.183 INFO:teuthology.orchestra.run.smithi027.stdout:(79/146): libtool-2.4.6-25.el8.x86_64.rpm 2.9 MB/s | 709 kB 00:00 2023-12-17T11:54:28.217 INFO:teuthology.orchestra.run.smithi141.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-12-17T11:54:28.284 INFO:teuthology.orchestra.run.smithi141.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 886 kB/s | 59 kB 00:00 2023-12-17T11:54:28.313 INFO:teuthology.orchestra.run.smithi027.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 300 kB/s | 39 kB 00:00 2023-12-17T11:54:28.360 INFO:teuthology.orchestra.run.smithi141.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 805 kB/s | 60 kB 00:00 2023-12-17T11:54:28.438 INFO:teuthology.orchestra.run.smithi027.stdout:(81/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-12-17T11:54:28.470 INFO:teuthology.orchestra.run.smithi192.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2023-12-17T11:54:28.537 INFO:teuthology.orchestra.run.smithi192.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 886 kB/s | 59 kB 00:00 2023-12-17T11:54:28.606 INFO:teuthology.orchestra.run.smithi027.stdout:(82/146): oniguruma-6.8.2-2.el8.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2023-12-17T11:54:28.621 INFO:teuthology.orchestra.run.smithi192.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 902 kB/s | 75 kB 00:00 2023-12-17T11:54:28.635 INFO:teuthology.orchestra.run.smithi141.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2023-12-17T11:54:28.705 INFO:teuthology.orchestra.run.smithi192.stdout:(40/146): luarocks-3.5.0-1.el8.noarch.rpm 2.0 MB/s | 167 kB 00:00 2023-12-17T11:54:28.710 INFO:teuthology.orchestra.run.smithi141.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2023-12-17T11:54:28.764 INFO:teuthology.orchestra.run.smithi027.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 405 kB/s | 64 kB 00:00 2023-12-17T11:54:28.786 INFO:teuthology.orchestra.run.smithi141.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 449 kB/s | 34 kB 00:00 2023-12-17T11:54:28.830 INFO:teuthology.orchestra.run.smithi192.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2023-12-17T11:54:28.897 INFO:teuthology.orchestra.run.smithi192.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 658 kB/s | 44 kB 00:00 2023-12-17T11:54:28.914 INFO:teuthology.orchestra.run.smithi027.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 691 kB/s | 103 kB 00:00 2023-12-17T11:54:28.961 INFO:teuthology.orchestra.run.smithi141.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-12-17T11:54:28.981 INFO:teuthology.orchestra.run.smithi192.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-12-17T11:54:29.056 INFO:teuthology.orchestra.run.smithi192.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 442 kB/s | 33 kB 00:00 2023-12-17T11:54:29.086 INFO:teuthology.orchestra.run.smithi141.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 514 kB/s | 64 kB 00:00 2023-12-17T11:54:29.123 INFO:teuthology.orchestra.run.smithi192.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 245 kB/s | 16 kB 00:00 2023-12-17T11:54:29.161 INFO:teuthology.orchestra.run.smithi141.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 250 kB/s | 19 kB 00:00 2023-12-17T11:54:29.190 INFO:teuthology.orchestra.run.smithi192.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-12-17T11:54:29.198 INFO:teuthology.orchestra.run.smithi027.stdout:(85/146): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-12-17T11:54:29.228 INFO:teuthology.orchestra.run.smithi141.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 361 kB/s | 24 kB 00:00 2023-12-17T11:54:29.298 INFO:teuthology.orchestra.run.smithi192.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-12-17T11:54:29.304 INFO:teuthology.orchestra.run.smithi141.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-12-17T11:54:29.374 INFO:teuthology.orchestra.run.smithi192.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-12-17T11:54:29.396 INFO:teuthology.orchestra.run.smithi141.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 300 kB/s | 27 kB 00:00 2023-12-17T11:54:29.482 INFO:teuthology.orchestra.run.smithi192.stdout:(49/146): ceph-mgr-diskprediction-local-19.0.0- 1.7 MB/s | 7.4 MB 00:04 2023-12-17T11:54:29.490 INFO:teuthology.orchestra.run.smithi027.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 1.8 MB/s | 538 kB 00:00 2023-12-17T11:54:29.508 INFO:teuthology.orchestra.run.smithi192.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-12-17T11:54:29.541 INFO:teuthology.orchestra.run.smithi192.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 301 kB/s | 18 kB 00:00 2023-12-17T11:54:29.596 INFO:teuthology.orchestra.run.smithi141.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-12-17T11:54:29.600 INFO:teuthology.orchestra.run.smithi192.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 838 kB/s | 49 kB 00:00 2023-12-17T11:54:29.683 INFO:teuthology.orchestra.run.smithi192.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 706 kB/s | 59 kB 00:00 2023-12-17T11:54:29.696 INFO:teuthology.orchestra.run.smithi141.stdout:(65/146): ceph-mgr-diskprediction-local-19.0.0- 925 kB/s | 7.4 MB 00:08 2023-12-17T11:54:29.724 INFO:teuthology.orchestra.run.smithi027.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 16 MB/s | 3.7 MB 00:00 2023-12-17T11:54:29.742 INFO:teuthology.orchestra.run.smithi192.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-12-17T11:54:29.809 INFO:teuthology.orchestra.run.smithi192.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-12-17T11:54:29.838 INFO:teuthology.orchestra.run.smithi141.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.0 MB/s | 251 kB 00:00 2023-12-17T11:54:29.851 INFO:teuthology.orchestra.run.smithi192.stdout:(56/146): python3-kubernetes-11.0.0-6.el8.noarc 3.4 MB/s | 1.2 MB 00:00 2023-12-17T11:54:29.877 INFO:teuthology.orchestra.run.smithi192.stdout:(57/146): python3-portend-2.6-1.el8.noarch.rpm 232 kB/s | 16 kB 00:00 2023-12-17T11:54:29.907 INFO:teuthology.orchestra.run.smithi027.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2023-12-17T11:54:29.910 INFO:teuthology.orchestra.run.smithi192.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 576 kB/s | 34 kB 00:00 2023-12-17T11:54:29.936 INFO:teuthology.orchestra.run.smithi192.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2023-12-17T11:54:29.955 INFO:teuthology.orchestra.run.smithi141.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 734 kB/s | 85 kB 00:00 2023-12-17T11:54:29.969 INFO:teuthology.orchestra.run.smithi192.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-12-17T11:54:29.981 INFO:teuthology.orchestra.run.smithi141.stdout:(68/146): python3-websocket-client-0.56.0-5.el8 216 kB/s | 61 kB 00:00 2023-12-17T11:54:30.008 INFO:teuthology.orchestra.run.smithi027.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-12-17T11:54:30.028 INFO:teuthology.orchestra.run.smithi192.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2023-12-17T11:54:30.031 INFO:teuthology.orchestra.run.smithi141.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-12-17T11:54:30.053 INFO:teuthology.orchestra.run.smithi192.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2023-12-17T11:54:30.056 INFO:teuthology.orchestra.run.smithi141.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-12-17T11:54:30.095 INFO:teuthology.orchestra.run.smithi192.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 439 kB/s | 29 kB 00:00 2023-12-17T11:54:30.120 INFO:teuthology.orchestra.run.smithi192.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2023-12-17T11:54:30.170 INFO:teuthology.orchestra.run.smithi192.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-12-17T11:54:30.196 INFO:teuthology.orchestra.run.smithi192.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-12-17T11:54:30.207 INFO:teuthology.orchestra.run.smithi141.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 119 kB/s | 18 kB 00:00 2023-12-17T11:54:30.238 INFO:teuthology.orchestra.run.smithi192.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 919 kB/s | 61 kB 00:00 2023-12-17T11:54:30.241 INFO:teuthology.orchestra.run.smithi027.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2023-12-17T11:54:30.263 INFO:teuthology.orchestra.run.smithi192.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-12-17T11:54:30.305 INFO:teuthology.orchestra.run.smithi192.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 344 kB/s | 23 kB 00:00 2023-12-17T11:54:30.361 INFO:teuthology.orchestra.run.smithi192.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-12-17T11:54:30.365 INFO:teuthology.orchestra.run.smithi141.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 270 kB/s | 43 kB 00:00 2023-12-17T11:54:30.456 INFO:teuthology.orchestra.run.smithi192.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 9.0 MB/s | 1.7 MB 00:00 2023-12-17T11:54:30.517 INFO:teuthology.orchestra.run.smithi027.stdout:(91/146): openblas-threads-0.3.15-6.el8.x86_64. 19 MB/s | 5.2 MB 00:00 2023-12-17T11:54:30.532 INFO:teuthology.orchestra.run.smithi141.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 931 kB/s | 155 kB 00:00 2023-12-17T11:54:30.548 INFO:teuthology.orchestra.run.smithi192.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 107 kB/s | 20 kB 00:00 2023-12-17T11:54:30.665 INFO:teuthology.orchestra.run.smithi192.stdout:(73/146): python3-pyasn1-modules-0.3.7-6.el8.no 946 kB/s | 110 kB 00:00 2023-12-17T11:54:30.732 INFO:teuthology.orchestra.run.smithi192.stdout:(74/146): python3-prettytable-0.7.2-14.el8.noar 159 kB/s | 44 kB 00:00 2023-12-17T11:54:30.741 INFO:teuthology.orchestra.run.smithi141.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 524 kB/s | 109 kB 00:00 2023-12-17T11:54:30.801 INFO:teuthology.orchestra.run.smithi027.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 554 kB/s | 157 kB 00:00 2023-12-17T11:54:30.857 INFO:teuthology.orchestra.run.smithi192.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2023-12-17T11:54:30.874 INFO:teuthology.orchestra.run.smithi192.stdout:(76/146): python3-jsonpatch-1.21-2.el8.noarch.r 127 kB/s | 27 kB 00:00 2023-12-17T11:54:30.874 INFO:teuthology.orchestra.run.smithi141.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 321 kB/s | 43 kB 00:00 2023-12-17T11:54:30.932 INFO:teuthology.orchestra.run.smithi192.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 6.0 MB/s | 457 kB 00:00 2023-12-17T11:54:31.000 INFO:teuthology.orchestra.run.smithi141.stdout:(76/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 150 kB/s | 19 kB 00:00 2023-12-17T11:54:31.091 INFO:teuthology.orchestra.run.smithi192.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 3.1 MB/s | 688 kB 00:00 2023-12-17T11:54:31.125 INFO:teuthology.orchestra.run.smithi192.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2023-12-17T11:54:31.183 INFO:teuthology.orchestra.run.smithi141.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 239 kB/s | 44 kB 00:00 2023-12-17T11:54:31.201 INFO:teuthology.orchestra.run.smithi027.stdout:(93/146): openblas-0.3.15-6.el8.x86_64.rpm 13 MB/s | 5.0 MB 00:00 2023-12-17T11:54:31.225 INFO:teuthology.orchestra.run.smithi192.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 388 kB/s | 39 kB 00:00 2023-12-17T11:54:31.258 INFO:teuthology.orchestra.run.smithi192.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2023-12-17T11:54:31.300 INFO:teuthology.orchestra.run.smithi141.stdout:(78/146): thrift-0.13.0-2.el8.x86_64.rpm 1.3 MB/s | 1.7 MB 00:01 2023-12-17T11:54:31.359 INFO:teuthology.orchestra.run.smithi141.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.4 MB/s | 605 kB 00:00 2023-12-17T11:54:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 3.3 MB/s | 714 kB 00:00 2023-12-17T11:54:31.434 INFO:teuthology.orchestra.run.smithi192.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2023-12-17T11:54:31.459 INFO:teuthology.orchestra.run.smithi141.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2023-12-17T11:54:31.476 INFO:teuthology.orchestra.run.smithi192.stdout:(83/146): oniguruma-6.8.2-2.el8.x86_64.rpm 864 kB/s | 187 kB 00:00 2023-12-17T11:54:31.560 INFO:teuthology.orchestra.run.smithi192.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 510 kB/s | 64 kB 00:00 2023-12-17T11:54:31.560 INFO:teuthology.orchestra.run.smithi141.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 2.0 MB/s | 200 kB 00:00 2023-12-17T11:54:31.585 INFO:teuthology.orchestra.run.smithi027.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-12-17T11:54:31.635 INFO:teuthology.orchestra.run.smithi141.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 800 kB/s | 220 kB 00:00 2023-12-17T11:54:31.677 INFO:teuthology.orchestra.run.smithi027.stdout:(96/146): ceph-mgr-diskprediction-local-19.0.0- 813 kB/s | 7.4 MB 00:09 2023-12-17T11:54:31.693 INFO:teuthology.orchestra.run.smithi192.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 477 kB/s | 103 kB 00:00 2023-12-17T11:54:31.727 INFO:teuthology.orchestra.run.smithi141.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 288 kB/s | 48 kB 00:00 2023-12-17T11:54:31.727 INFO:teuthology.orchestra.run.smithi027.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-12-17T11:54:31.777 INFO:teuthology.orchestra.run.smithi141.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 613 kB/s | 87 kB 00:00 2023-12-17T11:54:31.811 INFO:teuthology.orchestra.run.smithi027.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 407 kB/s | 54 kB 00:00 2023-12-17T11:54:31.818 INFO:teuthology.orchestra.run.smithi192.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 4.2 MB/s | 538 kB 00:00 2023-12-17T11:54:31.910 INFO:teuthology.orchestra.run.smithi192.stdout:(87/146): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-12-17T11:54:31.911 INFO:teuthology.orchestra.run.smithi027.stdout:(99/146): postgresql-10.23-3.module+el8.9.0+208 8.2 MB/s | 1.5 MB 00:00 2023-12-17T11:54:31.919 INFO:teuthology.orchestra.run.smithi141.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 301 kB/s | 58 kB 00:00 2023-12-17T11:54:31.944 INFO:teuthology.orchestra.run.smithi141.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 231 kB/s | 39 kB 00:00 2023-12-17T11:54:32.045 INFO:teuthology.orchestra.run.smithi027.stdout:(100/146): postgresql-contrib-10.23-3.module+el 3.4 MB/s | 811 kB 00:00 2023-12-17T11:54:32.094 INFO:teuthology.orchestra.run.smithi141.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 555 kB/s | 83 kB 00:00 2023-12-17T11:54:32.128 INFO:teuthology.orchestra.run.smithi192.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-12-17T11:54:32.153 INFO:teuthology.orchestra.run.smithi141.stdout:(88/146): openssl-devel-1.1.1k-9.el8_7.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2023-12-17T11:54:32.169 INFO:teuthology.orchestra.run.smithi192.stdout:(89/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.2 MB/s | 323 kB 00:00 2023-12-17T11:54:32.245 INFO:teuthology.orchestra.run.smithi027.stdout:(101/146): postgresql-server-10.23-3.module+el8 15 MB/s | 5.1 MB 00:00 2023-12-17T11:54:32.304 INFO:teuthology.orchestra.run.smithi141.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.0 MB/s | 645 kB 00:00 2023-12-17T11:54:32.329 INFO:teuthology.orchestra.run.smithi141.stdout:(90/146): libquadmath-8.5.0-20.el8.x86_64.rpm 983 kB/s | 172 kB 00:00 2023-12-17T11:54:32.353 INFO:teuthology.orchestra.run.smithi192.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.9 MB/s | 354 kB 00:00 2023-12-17T11:54:32.378 INFO:teuthology.orchestra.run.smithi192.stdout:(91/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 898 kB/s | 225 kB 00:00 2023-12-17T11:54:32.413 INFO:teuthology.orchestra.run.smithi141.stdout:(91/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 921 kB/s | 100 kB 00:00 2023-12-17T11:54:32.420 INFO:teuthology.orchestra.run.smithi027.stdout:(102/146): libverto-devel-0.3.0-5.el8.x86_64.rp 102 kB/s | 18 kB 00:00 2023-12-17T11:54:32.455 INFO:teuthology.orchestra.run.smithi141.stdout:(92/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2023-12-17T11:54:32.512 INFO:teuthology.orchestra.run.smithi192.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2023-12-17T11:54:32.546 INFO:teuthology.orchestra.run.smithi141.stdout:(93/146): python3-cryptography-3.2.1-6.el8.x86_ 4.1 MB/s | 559 kB 00:00 2023-12-17T11:54:32.572 INFO:teuthology.orchestra.run.smithi141.stdout:(94/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 622 kB/s | 73 kB 00:00 2023-12-17T11:54:32.647 INFO:teuthology.orchestra.run.smithi141.stdout:(95/146): NetworkManager-initscripts-updown-1.4 1.4 MB/s | 143 kB 00:00 2023-12-17T11:54:32.654 INFO:teuthology.orchestra.run.smithi192.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 17 MB/s | 5.2 MB 00:00 2023-12-17T11:54:32.655 INFO:teuthology.orchestra.run.smithi027.stdout:(103/146): python3-scipy-1.0.0-21.module+el8.9. 22 MB/s | 14 MB 00:00 2023-12-17T11:54:32.688 INFO:teuthology.orchestra.run.smithi027.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 160 kB/s | 43 kB 00:00 2023-12-17T11:54:32.714 INFO:teuthology.orchestra.run.smithi141.stdout:(96/146): elfutils-libelf-devel-0.189-3.el8.x86 435 kB/s | 62 kB 00:00 2023-12-17T11:54:32.781 INFO:teuthology.orchestra.run.smithi141.stdout:(97/146): librabbitmq-0.9.0-4.el8.x86_64.rpm 352 kB/s | 47 kB 00:00 2023-12-17T11:54:32.788 INFO:teuthology.orchestra.run.smithi027.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-12-17T11:54:32.855 INFO:teuthology.orchestra.run.smithi192.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2023-12-17T11:54:32.871 INFO:teuthology.orchestra.run.smithi027.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 593 kB/s | 109 kB 00:00 2023-12-17T11:54:32.881 INFO:teuthology.orchestra.run.smithi141.stdout:(98/146): python3-jsonpointer-1.10-11.el8.noarc 200 kB/s | 20 kB 00:00 2023-12-17T11:54:32.896 INFO:teuthology.orchestra.run.smithi192.stdout:(95/146): automake-1.16.1-8.el8.noarch.rpm 2.9 MB/s | 714 kB 00:00 2023-12-17T11:54:32.930 INFO:teuthology.orchestra.run.smithi027.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 301 kB/s | 43 kB 00:00 2023-12-17T11:54:32.956 INFO:teuthology.orchestra.run.smithi141.stdout:(99/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 2.3 MB/s | 562 kB 00:00 2023-12-17T11:54:32.997 INFO:teuthology.orchestra.run.smithi027.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2023-12-17T11:54:33.015 INFO:teuthology.orchestra.run.smithi141.stdout:(100/146): python3-prettytable-0.7.2-14.el8.noa 329 kB/s | 44 kB 00:00 2023-12-17T11:54:33.047 INFO:teuthology.orchestra.run.smithi027.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 375 kB/s | 44 kB 00:00 2023-12-17T11:54:33.054 INFO:teuthology.orchestra.run.smithi192.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 990 kB/s | 198 kB 00:00 2023-12-17T11:54:33.130 INFO:teuthology.orchestra.run.smithi192.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 867 kB/s | 202 kB 00:00 2023-12-17T11:54:33.173 INFO:teuthology.orchestra.run.smithi027.stdout:(110/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2023-12-17T11:54:33.192 INFO:teuthology.orchestra.run.smithi141.stdout:(101/146): python3-pyasn1-modules-0.3.7-6.el8.n 468 kB/s | 110 kB 00:00 2023-12-17T11:54:33.288 INFO:teuthology.orchestra.run.smithi192.stdout:(98/146): postgresql-10.23-3.module+el8.9.0+208 9.5 MB/s | 1.5 MB 00:00 2023-12-17T11:54:33.309 INFO:teuthology.orchestra.run.smithi027.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 1.9 MB/s | 605 kB 00:00 2023-12-17T11:54:33.341 INFO:teuthology.orchestra.run.smithi141.stdout:(102/146): python3-jsonpatch-1.21-2.el8.noarch. 82 kB/s | 27 kB 00:00 2023-12-17T11:54:33.397 INFO:teuthology.orchestra.run.smithi192.stdout:(99/146): postgresql-contrib-10.23-3.module+el8 7.3 MB/s | 811 kB 00:00 2023-12-17T11:54:33.464 INFO:teuthology.orchestra.run.smithi192.stdout:(100/146): python3-pytz-2017.2-11.el8.noarch.rp 133 kB/s | 54 kB 00:00 2023-12-17T11:54:33.465 INFO:teuthology.orchestra.run.smithi027.stdout:(112/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2023-12-17T11:54:33.516 INFO:teuthology.orchestra.run.smithi141.stdout:(103/146): python3-werkzeug-0.12.2-4.el8.noarch 2.6 MB/s | 457 kB 00:00 2023-12-17T11:54:33.616 INFO:teuthology.orchestra.run.smithi027.stdout:(113/146): keyutils-libs-devel-1.5.10-9.el8.x86 320 kB/s | 48 kB 00:00 2023-12-17T11:54:33.649 INFO:teuthology.orchestra.run.smithi027.stdout:(114/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 463 kB/s | 220 kB 00:00 2023-12-17T11:54:33.675 INFO:teuthology.orchestra.run.smithi141.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 262 kB/s | 126 kB 00:00 2023-12-17T11:54:33.708 INFO:teuthology.orchestra.run.smithi141.stdout:(105/146): bison-3.0.4-10.el8.x86_64.rpm 3.5 MB/s | 688 kB 00:00 2023-12-17T11:54:33.717 INFO:teuthology.orchestra.run.smithi027.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 872 kB/s | 87 kB 00:00 2023-12-17T11:54:33.749 INFO:teuthology.orchestra.run.smithi027.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 575 kB/s | 58 kB 00:00 2023-12-17T11:54:33.756 INFO:teuthology.orchestra.run.smithi192.stdout:(101/146): postgresql-server-10.23-3.module+el8 14 MB/s | 5.1 MB 00:00 2023-12-17T11:54:33.817 INFO:teuthology.orchestra.run.smithi141.stdout:(106/146): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2023-12-17T11:54:33.851 INFO:teuthology.orchestra.run.smithi141.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2023-12-17T11:54:33.858 INFO:teuthology.orchestra.run.smithi027.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 272 kB/s | 39 kB 00:00 2023-12-17T11:54:33.951 INFO:teuthology.orchestra.run.smithi141.stdout:(108/146): python3-markupsafe-0.23-19.el8.x86_6 291 kB/s | 39 kB 00:00 2023-12-17T11:54:33.998 INFO:teuthology.orchestra.run.smithi192.stdout:(102/146): python3-scipy-1.0.0-21.module+el8.9. 26 MB/s | 14 MB 00:00 2023-12-17T11:54:34.001 INFO:teuthology.orchestra.run.smithi141.stdout:(109/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2023-12-17T11:54:34.023 INFO:teuthology.orchestra.run.smithi192.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 67 kB/s | 18 kB 00:00 2023-12-17T11:54:34.033 INFO:teuthology.orchestra.run.smithi027.stdout:(118/146): openssl-devel-1.1.1k-9.el8_7.x86_64. 8.2 MB/s | 2.3 MB 00:00 2023-12-17T11:54:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 415 kB/s | 83 kB 00:00 2023-12-17T11:54:34.068 INFO:teuthology.orchestra.run.smithi141.stdout:(110/146): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2023-12-17T11:54:34.126 INFO:teuthology.orchestra.run.smithi141.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 511 kB/s | 64 kB 00:00 2023-12-17T11:54:34.132 INFO:teuthology.orchestra.run.smithi192.stdout:(104/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.4 MB/s | 155 kB 00:00 2023-12-17T11:54:34.157 INFO:teuthology.orchestra.run.smithi192.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 269 kB/s | 43 kB 00:00 2023-12-17T11:54:34.167 INFO:teuthology.orchestra.run.smithi027.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.7 MB/s | 645 kB 00:00 2023-12-17T11:54:34.178 INFO:teuthology.orchestra.run.smithi141.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 952 kB/s | 103 kB 00:00 2023-12-17T11:54:34.193 INFO:teuthology.orchestra.run.smithi027.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2023-12-17T11:54:34.274 INFO:teuthology.orchestra.run.smithi192.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 769 kB/s | 109 kB 00:00 2023-12-17T11:54:34.276 INFO:teuthology.orchestra.run.smithi027.stdout:(122/146): libuuid-devel-2.32.1-43.el8.x86_64.r 921 kB/s | 100 kB 00:00 2023-12-17T11:54:34.299 INFO:teuthology.orchestra.run.smithi192.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 301 kB/s | 43 kB 00:00 2023-12-17T11:54:34.310 INFO:teuthology.orchestra.run.smithi027.stdout:(123/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2023-12-17T11:54:34.352 INFO:teuthology.orchestra.run.smithi141.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-12-17T11:54:34.369 INFO:teuthology.orchestra.run.smithi027.stdout:(124/146): python3-cryptography-3.2.1-6.el8.x86 5.9 MB/s | 559 kB 00:00 2023-12-17T11:54:34.386 INFO:teuthology.orchestra.run.smithi141.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.5 MB/s | 538 kB 00:00 2023-12-17T11:54:34.427 INFO:teuthology.orchestra.run.smithi027.stdout:(125/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 623 kB/s | 73 kB 00:00 2023-12-17T11:54:34.433 INFO:teuthology.orchestra.run.smithi192.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 118 kB/s | 19 kB 00:00 2023-12-17T11:54:34.475 INFO:teuthology.orchestra.run.smithi192.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 251 kB/s | 44 kB 00:00 2023-12-17T11:54:34.494 INFO:teuthology.orchestra.run.smithi141.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-12-17T11:54:34.511 INFO:teuthology.orchestra.run.smithi027.stdout:(126/146): NetworkManager-initscripts-updown-1. 1.0 MB/s | 143 kB 00:00 2023-12-17T11:54:34.537 INFO:teuthology.orchestra.run.smithi027.stdout:(127/146): elfutils-libelf-devel-0.189-3.el8.x8 568 kB/s | 62 kB 00:00 2023-12-17T11:54:34.603 INFO:teuthology.orchestra.run.smithi141.stdout:(116/146): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-12-17T11:54:34.605 INFO:teuthology.orchestra.run.smithi192.stdout:(110/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2023-12-17T11:54:34.620 INFO:teuthology.orchestra.run.smithi027.stdout:(128/146): librabbitmq-0.9.0-4.el8.x86_64.rpm 433 kB/s | 47 kB 00:00 2023-12-17T11:54:34.628 INFO:teuthology.orchestra.run.smithi141.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2023-12-17T11:54:34.642 INFO:teuthology.orchestra.run.smithi192.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2023-12-17T11:54:34.670 INFO:teuthology.orchestra.run.smithi027.stdout:(129/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.1 MB/s | 562 kB 00:00 2023-12-17T11:54:34.676 INFO:teuthology.orchestra.run.smithi192.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.1 MB/s | 220 kB 00:00 2023-12-17T11:54:34.762 INFO:teuthology.orchestra.run.smithi141.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-12-17T11:54:34.793 INFO:teuthology.orchestra.run.smithi192.stdout:(113/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2023-12-17T11:54:34.885 INFO:teuthology.orchestra.run.smithi192.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 231 kB/s | 48 kB 00:00 2023-12-17T11:54:34.904 INFO:teuthology.orchestra.run.smithi141.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 19 MB/s | 5.2 MB 00:00 2023-12-17T11:54:34.929 INFO:teuthology.orchestra.run.smithi141.stdout:(120/146): python3-mako-1.0.6-14.el8.noarch.rpm 941 kB/s | 157 kB 00:00 2023-12-17T11:54:34.985 INFO:teuthology.orchestra.run.smithi192.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 453 kB/s | 87 kB 00:00 2023-12-17T11:54:35.019 INFO:teuthology.orchestra.run.smithi192.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 431 kB/s | 58 kB 00:00 2023-12-17T11:54:35.080 INFO:teuthology.orchestra.run.smithi141.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 4.6 MB/s | 714 kB 00:00 2023-12-17T11:54:35.094 INFO:teuthology.orchestra.run.smithi192.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 356 kB/s | 39 kB 00:00 2023-12-17T11:54:35.205 INFO:teuthology.orchestra.run.smithi141.stdout:(122/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2023-12-17T11:54:35.230 INFO:teuthology.orchestra.run.smithi141.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.3 MB/s | 198 kB 00:00 2023-12-17T11:54:35.308 INFO:teuthology.orchestra.run.smithi192.stdout:(118/146): openssl-devel-1.1.1k-9.el8_7.x86_64. 8.1 MB/s | 2.3 MB 00:00 2023-12-17T11:54:35.345 INFO:teuthology.orchestra.run.smithi192.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 332 kB/s | 83 kB 00:00 2023-12-17T11:54:35.355 INFO:teuthology.orchestra.run.smithi141.stdout:(124/146): python3-pytz-2017.2-11.el8.noarch.rp 434 kB/s | 54 kB 00:00 2023-12-17T11:54:35.406 INFO:teuthology.orchestra.run.smithi141.stdout:(125/146): jq-1.6-7.el8.x86_64.rpm 1.0 MB/s | 202 kB 00:00 2023-12-17T11:54:35.470 INFO:teuthology.orchestra.run.smithi192.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2023-12-17T11:54:35.514 INFO:teuthology.orchestra.run.smithi141.stdout:(126/146): postgresql-contrib-10.23-3.module+el 7.3 MB/s | 811 kB 00:00 2023-12-17T11:54:35.520 INFO:teuthology.orchestra.run.smithi192.stdout:(121/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.0 MB/s | 645 kB 00:00 2023-12-17T11:54:35.581 INFO:teuthology.orchestra.run.smithi141.stdout:(127/146): postgresql-10.23-3.module+el8.9.0+20 6.7 MB/s | 1.5 MB 00:00 2023-12-17T11:54:35.671 INFO:teuthology.orchestra.run.smithi192.stdout:(122/146): libuuid-devel-2.32.1-43.el8.x86_64.r 497 kB/s | 100 kB 00:00 2023-12-17T11:54:35.712 INFO:teuthology.orchestra.run.smithi192.stdout:(123/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-12-17T11:54:35.804 INFO:teuthology.orchestra.run.smithi192.stdout:(124/146): python3-cryptography-3.2.1-6.el8.x86 4.1 MB/s | 559 kB 00:00 2023-12-17T11:54:35.823 INFO:teuthology.orchestra.run.smithi141.stdout:(128/146): postgresql-server-10.23-3.module+el8 16 MB/s | 5.1 MB 00:00 2023-12-17T11:54:35.896 INFO:teuthology.orchestra.run.smithi192.stdout:(125/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 396 kB/s | 73 kB 00:00 2023-12-17T11:54:36.046 INFO:teuthology.orchestra.run.smithi192.stdout:(126/146): NetworkManager-initscripts-updown-1. 594 kB/s | 143 kB 00:00 2023-12-17T11:54:36.163 INFO:teuthology.orchestra.run.smithi192.stdout:(127/146): librabbitmq-0.9.0-4.el8.x86_64.rpm 403 kB/s | 47 kB 00:00 2023-12-17T11:54:36.197 INFO:teuthology.orchestra.run.smithi192.stdout:(128/146): elfutils-libelf-devel-0.189-3.el8.x8 205 kB/s | 62 kB 00:00 2023-12-17T11:54:36.215 INFO:teuthology.orchestra.run.smithi141.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 22 MB/s | 14 MB 00:00 2023-12-17T11:54:36.331 INFO:teuthology.orchestra.run.smithi192.stdout:(129/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.3 MB/s | 562 kB 00:00 2023-12-17T11:54:38.426 INFO:teuthology.orchestra.run.smithi027.stdout:(130/146): librados2-19.0.0-119.ga62fda73.el8.x 1.0 MB/s | 3.8 MB 00:03 2023-12-17T11:54:38.587 INFO:teuthology.orchestra.run.smithi027.stdout:(131/146): openssl-1.1.1k-9.el8_7.x86_64.rpm 4.3 MB/s | 710 kB 00:00 2023-12-17T11:54:38.746 INFO:teuthology.orchestra.run.smithi027.stdout:(132/146): openssl-libs-1.1.1k-9.el8_7.x86_64.r 9.3 MB/s | 1.5 MB 00:00 2023-12-17T11:54:38.879 INFO:teuthology.orchestra.run.smithi027.stdout:(133/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2023-12-17T11:54:39.022 INFO:teuthology.orchestra.run.smithi192.stdout:(130/146): librbd1-19.0.0-119.ga62fda73.el8.x86 1.4 MB/s | 3.7 MB 00:02 2023-12-17T11:54:39.080 INFO:teuthology.orchestra.run.smithi027.stdout:(134/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2023-12-17T11:54:39.156 INFO:teuthology.orchestra.run.smithi192.stdout:(131/146): openssl-1.1.1k-9.el8_7.x86_64.rpm 5.2 MB/s | 710 kB 00:00 2023-12-17T11:54:39.213 INFO:teuthology.orchestra.run.smithi027.stdout:(135/146): libmount-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 237 kB 00:00 2023-12-17T11:54:39.248 INFO:teuthology.orchestra.run.smithi141.stdout:(130/146): ceph-test-19.0.0-119.ga62fda73.el8.x 1.5 MB/s | 49 MB 00:33 2023-12-17T11:54:39.272 INFO:teuthology.orchestra.run.smithi027.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 3.0 MB/s | 179 kB 00:00 2023-12-17T11:54:39.290 INFO:teuthology.orchestra.run.smithi192.stdout:(132/146): openssl-libs-1.1.1k-9.el8_7.x86_64.r 11 MB/s | 1.5 MB 00:00 2023-12-17T11:54:39.407 INFO:teuthology.orchestra.run.smithi141.stdout:(131/146): openssl-1.1.1k-9.el8_7.x86_64.rpm 4.4 MB/s | 710 kB 00:00 2023-12-17T11:54:39.447 INFO:teuthology.orchestra.run.smithi027.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 566 kB/s | 99 kB 00:00 2023-12-17T11:54:39.473 INFO:teuthology.orchestra.run.smithi192.stdout:(133/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2023-12-17T11:54:39.632 INFO:teuthology.orchestra.run.smithi141.stdout:(132/146): librbd1-19.0.0-119.ga62fda73.el8.x86 1.1 MB/s | 3.7 MB 00:03 2023-12-17T11:54:39.657 INFO:teuthology.orchestra.run.smithi192.stdout:(134/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.6 MB/s | 303 kB 00:00 2023-12-17T11:54:39.664 INFO:teuthology.orchestra.run.smithi027.stdout:(138/146): util-linux-2.32.1-43.el8.x86_64.rpm 11 MB/s | 2.5 MB 00:00 2023-12-17T11:54:39.699 INFO:teuthology.orchestra.run.smithi141.stdout:(133/146): openssl-libs-1.1.1k-9.el8_7.x86_64.r 5.0 MB/s | 1.5 MB 00:00 2023-12-17T11:54:39.766 INFO:teuthology.orchestra.run.smithi141.stdout:(134/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2023-12-17T11:54:39.790 INFO:teuthology.orchestra.run.smithi192.stdout:(135/146): libmount-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 237 kB 00:00 2023-12-17T11:54:39.848 INFO:teuthology.orchestra.run.smithi027.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 559 kB/s | 103 kB 00:00 2023-12-17T11:54:39.849 INFO:teuthology.orchestra.run.smithi141.stdout:(135/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.0 MB/s | 303 kB 00:00 2023-12-17T11:54:39.857 INFO:teuthology.orchestra.run.smithi192.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 2.6 MB/s | 179 kB 00:00 2023-12-17T11:54:39.883 INFO:teuthology.orchestra.run.smithi141.stdout:(136/146): libmount-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 237 kB 00:00 2023-12-17T11:54:40.000 INFO:teuthology.orchestra.run.smithi141.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 850 kB/s | 99 kB 00:00 2023-12-17T11:54:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:(140/146): xfsdump-3.1.8-6.el8.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2023-12-17T11:54:40.030 INFO:teuthology.orchestra.run.smithi141.stdout:(138/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.0 MB/s | 179 kB 00:00 2023-12-17T11:54:40.083 INFO:teuthology.orchestra.run.smithi192.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 440 kB/s | 99 kB 00:00 2023-12-17T11:54:40.142 INFO:teuthology.orchestra.run.smithi141.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 882 kB/s | 103 kB 00:00 2023-12-17T11:54:40.210 INFO:teuthology.orchestra.run.smithi141.stdout:(140/146): util-linux-2.32.1-43.el8.x86_64.rpm 12 MB/s | 2.5 MB 00:00 2023-12-17T11:54:40.215 INFO:teuthology.orchestra.run.smithi027.stdout:(141/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.4 MB/s | 854 kB 00:00 2023-12-17T11:54:40.293 INFO:teuthology.orchestra.run.smithi141.stdout:(141/146): xfsdump-3.1.8-6.el8.x86_64.rpm 2.2 MB/s | 333 kB 00:00 2023-12-17T11:54:40.308 INFO:teuthology.orchestra.run.smithi192.stdout:(138/146): util-linux-2.32.1-43.el8.x86_64.rpm 11 MB/s | 2.5 MB 00:00 2023-12-17T11:54:40.410 INFO:teuthology.orchestra.run.smithi141.stdout:(142/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.2 MB/s | 854 kB 00:00 2023-12-17T11:54:40.441 INFO:teuthology.orchestra.run.smithi027.stdout:(142/146): libblkid-2.32.1-43.el8.x86_64.rpm 982 kB/s | 220 kB 00:00 2023-12-17T11:54:40.544 INFO:teuthology.orchestra.run.smithi141.stdout:(143/146): librados2-19.0.0-119.ga62fda73.el8.x 817 kB/s | 3.8 MB 00:04 2023-12-17T11:54:40.577 INFO:teuthology.orchestra.run.smithi141.stdout:(144/146): libblkid-2.32.1-43.el8.x86_64.rpm 777 kB/s | 220 kB 00:00 2023-12-17T11:54:40.591 INFO:teuthology.orchestra.run.smithi192.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 363 kB/s | 103 kB 00:00 2023-12-17T11:54:40.607 INFO:teuthology.orchestra.run.smithi027.stdout:(143/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 254 kB 00:00 2023-12-17T11:54:40.610 INFO:teuthology.orchestra.run.smithi141.stdout:(145/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 254 kB 00:00 2023-12-17T11:54:40.753 INFO:teuthology.orchestra.run.smithi141.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.1 MB/s | 1.1 MB 00:00 2023-12-17T11:54:40.756 INFO:teuthology.orchestra.run.smithi141.stdout:-------------------------------------------------------------------------------- 2023-12-17T11:54:40.756 INFO:teuthology.orchestra.run.smithi141.stdout:Total 3.3 MB/s | 218 MB 01:05 2023-12-17T11:54:40.800 INFO:teuthology.orchestra.run.smithi192.stdout:(140/146): xfsdump-3.1.8-6.el8.x86_64.rpm 1.6 MB/s | 333 kB 00:00 2023-12-17T11:54:40.908 INFO:teuthology.orchestra.run.smithi027.stdout:(144/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 3.6 MB/s | 1.1 MB 00:00 2023-12-17T11:54:41.009 INFO:teuthology.orchestra.run.smithi192.stdout:(141/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.0 MB/s | 854 kB 00:00 2023-12-17T11:54:41.068 INFO:teuthology.orchestra.run.smithi192.stdout:(142/146): librados2-19.0.0-119.ga62fda73.el8.x 792 kB/s | 3.8 MB 00:04 2023-12-17T11:54:41.143 INFO:teuthology.orchestra.run.smithi192.stdout:(143/146): libfdisk-2.32.1-43.el8.x86_64.rpm 3.3 MB/s | 254 kB 00:00 2023-12-17T11:54:41.176 INFO:teuthology.orchestra.run.smithi192.stdout:(144/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-12-17T11:54:41.268 INFO:teuthology.orchestra.run.smithi192.stdout:(145/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2023-12-17T11:54:41.316 INFO:teuthology.orchestra.run.smithi027.stdout:(145/146): librbd1-19.0.0-119.ga62fda73.el8.x86 572 kB/s | 3.7 MB 00:06 2023-12-17T11:54:42.792 INFO:teuthology.orchestra.run.smithi027.stdout:(146/146): ceph-test-19.0.0-119.ga62fda73.el8.x 1.0 MB/s | 49 MB 00:48 2023-12-17T11:54:42.795 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-12-17T11:54:42.795 INFO:teuthology.orchestra.run.smithi027.stdout:Total 3.4 MB/s | 218 MB 01:04 2023-12-17T11:54:42.891 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction check 2023-12-17T11:54:43.338 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction check succeeded. 2023-12-17T11:54:43.338 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction test 2023-12-17T11:54:45.016 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-12-17T11:54:45.025 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction test succeeded. 2023-12-17T11:54:45.037 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction 2023-12-17T11:54:45.501 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-12-17T11:54:45.501 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-12-17T11:54:47.273 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-12-17T11:54:47.286 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-12-17T11:54:47.696 INFO:teuthology.orchestra.run.smithi192.stdout:(146/146): ceph-test-19.0.0-119.ga62fda73.el8.x 1.0 MB/s | 49 MB 00:48 2023-12-17T11:54:47.699 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-12-17T11:54:47.699 INFO:teuthology.orchestra.run.smithi192.stdout:Total 3.2 MB/s | 218 MB 01:08 2023-12-17T11:54:49.819 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-12-17T11:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-12-17T11:54:50.266 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-12-17T11:54:50.629 INFO:teuthology.orchestra.run.smithi141.stdout: Preparing : 1/1 2023-12-17T11:54:50.881 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-17T11:54:51.171 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/162 2023-12-17T11:54:51.435 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/162 2023-12-17T11:54:51.633 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2023-12-17T11:54:51.820 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2023-12-17T11:54:51.918 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2023-12-17T11:54:51.971 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-12-17T11:54:51.983 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-12-17T11:54:52.103 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2023-12-17T11:54:52.197 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2023-12-17T11:54:52.425 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2023-12-17T11:54:52.622 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/162 2023-12-17T11:54:52.837 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2023-12-17T11:54:52.968 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2023-12-17T11:54:53.031 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-12-17T11:54:53.178 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2023-12-17T11:54:53.274 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-17T11:54:53.552 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/162 2023-12-17T11:54:53.634 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2023-12-17T11:54:53.870 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:54:53.895 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/162 2023-12-17T11:54:54.092 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2023-12-17T11:54:54.294 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2023-12-17T11:54:54.313 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:54:54.391 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2023-12-17T11:54:54.503 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:54:54.619 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2023-12-17T11:54:54.653 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:54:54.703 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2023-12-17T11:54:54.845 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:54:54.923 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2023-12-17T11:54:54.993 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:54:55.178 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2023-12-17T11:54:55.268 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2023-12-17T11:54:55.508 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/162 2023-12-17T11:54:55.845 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 12/162 2023-12-17T11:54:55.919 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2023-12-17T11:54:56.061 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:54:56.281 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:54:56.414 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2023-12-17T11:54:56.588 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/162 2023-12-17T11:54:56.644 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:54:56.703 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/162 2023-12-17T11:54:56.943 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libpq-13.11-1.el8.x86_64 15/162 2023-12-17T11:54:57.135 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:54:57.202 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 16/162 2023-12-17T11:54:57.330 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:54:57.391 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/162 2023-12-17T11:54:57.480 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:54:57.694 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:54:57.800 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:54:57.850 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 18/162 2023-12-17T11:54:57.893 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-12-17T11:54:58.111 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 19/162 2023-12-17T11:54:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-17T11:54:58.260 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 20/162 2023-12-17T11:54:58.398 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 20/162 2023-12-17T11:54:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/162 2023-12-17T11:54:58.544 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 21/162 2023-12-17T11:54:58.643 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 21/162 2023-12-17T11:54:58.706 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 12/162 2023-12-17T11:54:58.724 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/162 2023-12-17T11:54:58.833 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 22/162 2023-12-17T11:54:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2023-12-17T11:54:58.961 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:54:59.079 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/162 2023-12-17T11:54:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2023-12-17T11:54:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:54:59.236 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2023-12-17T11:54:59.289 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/162 2023-12-17T11:54:59.427 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 25/162 2023-12-17T11:54:59.473 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2023-12-17T11:54:59.502 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/162 2023-12-17T11:54:59.570 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2023-12-17T11:54:59.597 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 25/162 2023-12-17T11:54:59.620 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/162 2023-12-17T11:54:59.743 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 26/162 2023-12-17T11:54:59.850 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2023-12-17T11:54:59.891 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpq-13.11-1.el8.x86_64 15/162 2023-12-17T11:55:00.096 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2023-12-17T11:55:00.174 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 16/162 2023-12-17T11:55:00.193 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2023-12-17T11:55:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/162 2023-12-17T11:55:00.432 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 26/162 2023-12-17T11:55:00.434 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/162 2023-12-17T11:55:00.618 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 27/162 2023-12-17T11:55:00.631 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2023-12-17T11:55:00.869 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 18/162 2023-12-17T11:55:01.086 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2023-12-17T11:55:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 19/162 2023-12-17T11:55:01.269 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 20/162 2023-12-17T11:55:01.297 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 27/162 2023-12-17T11:55:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:55:01.402 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 20/162 2023-12-17T11:55:01.518 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : openblas-0.3.15-6.el8.x86_64 28/162 2023-12-17T11:55:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 21/162 2023-12-17T11:55:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 21/162 2023-12-17T11:55:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:55:01.868 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/162 2023-12-17T11:55:02.024 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/162 2023-12-17T11:55:02.072 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:55:02.201 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/162 2023-12-17T11:55:02.224 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:55:02.324 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 28/162 2023-12-17T11:55:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:55:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:55:02.576 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 29/162 2023-12-17T11:55:02.810 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 30/162 2023-12-17T11:55:02.938 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/162 2023-12-17T11:55:03.065 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 31/162 2023-12-17T11:55:03.110 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 24/162 2023-12-17T11:55:03.275 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 32/162 2023-12-17T11:55:03.414 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 33/162 2023-12-17T11:55:03.437 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 12/162 2023-12-17T11:55:03.647 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:55:03.670 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 34/162 2023-12-17T11:55:03.818 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 24/162 2023-12-17T11:55:03.868 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/162 2023-12-17T11:55:03.913 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:55:03.985 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.15-6.el8.x86_64 25/162 2023-12-17T11:55:04.202 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/162 2023-12-17T11:55:04.311 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/162 2023-12-17T11:55:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpq-13.11-1.el8.x86_64 15/162 2023-12-17T11:55:04.856 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 25/162 2023-12-17T11:55:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 16/162 2023-12-17T11:55:05.061 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/162 2023-12-17T11:55:05.101 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/162 2023-12-17T11:55:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 27/162 2023-12-17T11:55:05.485 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 28/162 2023-12-17T11:55:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 18/162 2023-12-17T11:55:05.764 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 19/162 2023-12-17T11:55:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/162 2023-12-17T11:55:05.929 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/162 2023-12-17T11:55:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 20/162 2023-12-17T11:55:06.075 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 20/162 2023-12-17T11:55:06.084 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 30/162 2023-12-17T11:55:06.268 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 36/162 2023-12-17T11:55:06.272 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 21/162 2023-12-17T11:55:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 31/162 2023-12-17T11:55:06.362 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 21/162 2023-12-17T11:55:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/162 2023-12-17T11:55:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 32/162 2023-12-17T11:55:06.705 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/162 2023-12-17T11:55:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 33/162 2023-12-17T11:55:06.830 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/162 2023-12-17T11:55:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 34/162 2023-12-17T11:55:07.393 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 37/162 2023-12-17T11:55:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/162 2023-12-17T11:55:07.746 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 24/162 2023-12-17T11:55:07.763 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 38/162 2023-12-17T11:55:08.000 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 39/162 2023-12-17T11:55:08.196 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/162 2023-12-17T11:55:08.229 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-rsa-4.9-2.el8.noarch 40/162 2023-12-17T11:55:08.415 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 24/162 2023-12-17T11:55:08.422 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/162 2023-12-17T11:55:08.548 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/162 2023-12-17T11:55:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.15-6.el8.x86_64 25/162 2023-12-17T11:55:08.615 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 42/162 2023-12-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 43/162 2023-12-17T11:55:08.850 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/162 2023-12-17T11:55:08.977 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 44/162 2023-12-17T11:55:09.051 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/162 2023-12-17T11:55:09.208 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/162 2023-12-17T11:55:09.306 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/162 2023-12-17T11:55:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 25/162 2023-12-17T11:55:09.543 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/162 2023-12-17T11:55:09.631 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/162 2023-12-17T11:55:09.671 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/162 2023-12-17T11:55:09.852 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/162 2023-12-17T11:55:09.863 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 27/162 2023-12-17T11:55:10.028 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 28/162 2023-12-17T11:55:10.078 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 48/162 2023-12-17T11:55:10.310 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 49/162 2023-12-17T11:55:10.330 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/162 2023-12-17T11:55:10.645 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 30/162 2023-12-17T11:55:10.913 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 49/162 2023-12-17T11:55:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 31/162 2023-12-17T11:55:11.103 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 32/162 2023-12-17T11:55:11.217 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 50/162 2023-12-17T11:55:11.292 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 33/162 2023-12-17T11:55:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/162 2023-12-17T11:55:11.340 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 50/162 2023-12-17T11:55:11.552 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 34/162 2023-12-17T11:55:11.563 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2023-12-17T11:55:11.636 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 39/162 2023-12-17T11:55:11.758 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2023-12-17T11:55:11.856 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2023-12-17T11:55:11.891 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el8.noarch 40/162 2023-12-17T11:55:11.991 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 53/162 2023-12-17T11:55:12.090 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/162 2023-12-17T11:55:12.097 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 53/162 2023-12-17T11:55:12.259 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : jq-1.6-7.el8.x86_64 54/162 2023-12-17T11:55:12.308 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 42/162 2023-12-17T11:55:12.450 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 55/162 2023-12-17T11:55:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 43/162 2023-12-17T11:55:12.609 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 56/162 2023-12-17T11:55:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/162 2023-12-17T11:55:12.783 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 57/162 2023-12-17T11:55:12.787 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 44/162 2023-12-17T11:55:13.004 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/162 2023-12-17T11:55:13.005 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : fuse-2.9.7-17.el8.x86_64 58/162 2023-12-17T11:55:13.009 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/162 2023-12-17T11:55:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/162 2023-12-17T11:55:13.204 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 59/162 2023-12-17T11:55:13.286 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/162 2023-12-17T11:55:13.365 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/162 2023-12-17T11:55:13.383 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 60/162 2023-12-17T11:55:13.454 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/162 2023-12-17T11:55:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/162 2023-12-17T11:55:13.555 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 61/162 2023-12-17T11:55:13.699 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/162 2023-12-17T11:55:13.747 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/162 2023-12-17T11:55:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 48/162 2023-12-17T11:55:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 49/162 2023-12-17T11:55:14.072 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 63/162 2023-12-17T11:55:14.394 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/162 2023-12-17T11:55:14.638 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/162 2023-12-17T11:55:14.724 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 49/162 2023-12-17T11:55:14.809 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 66/162 2023-12-17T11:55:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 50/162 2023-12-17T11:55:15.076 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 67/162 2023-12-17T11:55:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 50/162 2023-12-17T11:55:15.289 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.7-17.el8.x86_64 51/162 2023-12-17T11:55:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 52/162 2023-12-17T11:55:15.498 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 68/162 2023-12-17T11:55:15.540 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/162 2023-12-17T11:55:15.700 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/162 2023-12-17T11:55:15.772 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 69/162 2023-12-17T11:55:15.881 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/162 2023-12-17T11:55:15.910 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 39/162 2023-12-17T11:55:15.997 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 70/162 2023-12-17T11:55:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/162 2023-12-17T11:55:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el8.noarch 40/162 2023-12-17T11:55:16.252 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 71/162 2023-12-17T11:55:16.358 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/162 2023-12-17T11:55:16.493 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 56/162 2023-12-17T11:55:16.517 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2023-12-17T11:55:16.617 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 42/162 2023-12-17T11:55:16.762 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 57/162 2023-12-17T11:55:16.835 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 43/162 2023-12-17T11:55:16.882 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2023-12-17T11:55:17.048 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/162 2023-12-17T11:55:17.050 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 44/162 2023-12-17T11:55:17.109 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 74/162 2023-12-17T11:55:17.247 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 59/162 2023-12-17T11:55:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/162 2023-12-17T11:55:17.318 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 75/162 2023-12-17T11:55:17.392 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/162 2023-12-17T11:55:17.510 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 76/162 2023-12-17T11:55:17.549 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 60/162 2023-12-17T11:55:17.657 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/162 2023-12-17T11:55:17.707 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 77/162 2023-12-17T11:55:17.746 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/162 2023-12-17T11:55:17.958 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/162 2023-12-17T11:55:17.976 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 61/162 2023-12-17T11:55:18.010 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 78/162 2023-12-17T11:55:18.175 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 79/162 2023-12-17T11:55:18.183 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 48/162 2023-12-17T11:55:18.308 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 62/162 2023-12-17T11:55:18.373 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 80/162 2023-12-17T11:55:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 49/162 2023-12-17T11:55:18.542 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 63/162 2023-12-17T11:55:18.622 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 81/162 2023-12-17T11:55:18.840 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 82/162 2023-12-17T11:55:18.868 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2023-12-17T11:55:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 49/162 2023-12-17T11:55:19.037 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 83/162 2023-12-17T11:55:19.120 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2023-12-17T11:55:19.195 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 84/162 2023-12-17T11:55:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 50/162 2023-12-17T11:55:19.362 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 50/162 2023-12-17T11:55:19.421 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 85/162 2023-12-17T11:55:19.479 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2023-12-17T11:55:19.583 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.7-17.el8.x86_64 51/162 2023-12-17T11:55:19.673 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 86/162 2023-12-17T11:55:19.727 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 67/162 2023-12-17T11:55:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 52/162 2023-12-17T11:55:19.878 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 87/162 2023-12-17T11:55:19.920 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 68/162 2023-12-17T11:55:20.036 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 88/162 2023-12-17T11:55:20.043 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/162 2023-12-17T11:55:20.169 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/162 2023-12-17T11:55:20.188 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 89/162 2023-12-17T11:55:20.224 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/162 2023-12-17T11:55:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 70/162 2023-12-17T11:55:20.416 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/162 2023-12-17T11:55:20.430 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 90/162 2023-12-17T11:55:20.688 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 91/162 2023-12-17T11:55:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 71/162 2023-12-17T11:55:20.782 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 56/162 2023-12-17T11:55:20.855 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 92/162 2023-12-17T11:55:20.945 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2023-12-17T11:55:21.110 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 57/162 2023-12-17T11:55:21.114 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-portend-2.6-1.el8.noarch 93/162 2023-12-17T11:55:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2023-12-17T11:55:21.258 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2023-12-17T11:55:21.358 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 94/162 2023-12-17T11:55:21.367 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/162 2023-12-17T11:55:21.474 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 74/162 2023-12-17T11:55:21.537 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 59/162 2023-12-17T11:55:21.550 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 95/162 2023-12-17T11:55:21.583 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 74/162 2023-12-17T11:55:21.779 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 96/162 2023-12-17T11:55:21.785 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : jq-1.6-7.el8.x86_64 75/162 2023-12-17T11:55:21.829 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 60/162 2023-12-17T11:55:22.017 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 76/162 2023-12-17T11:55:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 77/162 2023-12-17T11:55:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 61/162 2023-12-17T11:55:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 78/162 2023-12-17T11:55:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 62/162 2023-12-17T11:55:22.628 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 79/162 2023-12-17T11:55:22.801 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 63/162 2023-12-17T11:55:22.828 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 80/162 2023-12-17T11:55:23.079 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 97/162 2023-12-17T11:55:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2023-12-17T11:55:23.144 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 81/162 2023-12-17T11:55:23.387 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2023-12-17T11:55:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 82/162 2023-12-17T11:55:23.426 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 98/162 2023-12-17T11:55:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 83/162 2023-12-17T11:55:23.669 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 99/162 2023-12-17T11:55:23.742 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2023-12-17T11:55:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 84/162 2023-12-17T11:55:23.970 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 67/162 2023-12-17T11:55:24.002 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 100/162 2023-12-17T11:55:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 85/162 2023-12-17T11:55:24.222 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 68/162 2023-12-17T11:55:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 86/162 2023-12-17T11:55:24.296 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 101/162 2023-12-17T11:55:24.480 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/162 2023-12-17T11:55:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 87/162 2023-12-17T11:55:24.546 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : luarocks-3.5.0-1.el8.noarch 102/162 2023-12-17T11:55:24.689 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 70/162 2023-12-17T11:55:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 88/162 2023-12-17T11:55:24.793 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 103/162 2023-12-17T11:55:24.891 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 89/162 2023-12-17T11:55:25.023 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 104/162 2023-12-17T11:55:25.106 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 71/162 2023-12-17T11:55:25.136 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 90/162 2023-12-17T11:55:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2023-12-17T11:55:25.395 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 91/162 2023-12-17T11:55:25.519 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2023-12-17T11:55:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 92/162 2023-12-17T11:55:25.626 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2023-12-17T11:55:25.794 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 74/162 2023-12-17T11:55:25.916 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 74/162 2023-12-17T11:55:25.922 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-2.6-1.el8.noarch 93/162 2023-12-17T11:55:26.103 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-7.el8.x86_64 75/162 2023-12-17T11:55:26.151 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 94/162 2023-12-17T11:55:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 76/162 2023-12-17T11:55:26.345 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 95/162 2023-12-17T11:55:26.494 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 77/162 2023-12-17T11:55:26.722 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 78/162 2023-12-17T11:55:26.888 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 79/162 2023-12-17T11:55:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 80/162 2023-12-17T11:55:27.154 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 96/162 2023-12-17T11:55:27.429 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 81/162 2023-12-17T11:55:27.695 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 82/162 2023-12-17T11:55:27.891 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 83/162 2023-12-17T11:55:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 84/162 2023-12-17T11:55:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 85/162 2023-12-17T11:55:28.521 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 97/162 2023-12-17T11:55:28.611 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 86/162 2023-12-17T11:55:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 98/162 2023-12-17T11:55:28.885 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 87/162 2023-12-17T11:55:29.098 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 88/162 2023-12-17T11:55:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 99/162 2023-12-17T11:55:29.251 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 89/162 2023-12-17T11:55:29.508 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 90/162 2023-12-17T11:55:29.509 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 100/162 2023-12-17T11:55:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 91/162 2023-12-17T11:55:29.845 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 101/162 2023-12-17T11:55:30.070 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.5.0-1.el8.noarch 102/162 2023-12-17T11:55:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 103/162 2023-12-17T11:55:30.425 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 92/162 2023-12-17T11:55:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 104/162 2023-12-17T11:55:30.665 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:30.738 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-2.6-1.el8.noarch 93/162 2023-12-17T11:55:31.028 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 94/162 2023-12-17T11:55:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 95/162 2023-12-17T11:55:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 96/162 2023-12-17T11:55:31.614 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:32.322 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:32.664 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 106/162 2023-12-17T11:55:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 97/162 2023-12-17T11:55:32.804 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 106/162 2023-12-17T11:55:32.971 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 107/162 2023-12-17T11:55:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 98/162 2023-12-17T11:55:33.439 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 99/162 2023-12-17T11:55:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 100/162 2023-12-17T11:55:34.116 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 101/162 2023-12-17T11:55:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.5.0-1.el8.noarch 102/162 2023-12-17T11:55:34.678 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 103/162 2023-12-17T11:55:34.927 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 104/162 2023-12-17T11:55:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:37.619 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 106/162 2023-12-17T11:55:38.994 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 106/162 2023-12-17T11:55:39.189 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 107/162 2023-12-17T11:55:40.404 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:40.635 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:41.335 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 105/162 2023-12-17T11:55:42.241 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 106/162 2023-12-17T11:55:42.389 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 106/162 2023-12-17T11:55:42.558 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 107/162 2023-12-17T11:55:54.405 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 107/162 2023-12-17T11:55:54.405 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /sys 2023-12-17T11:55:54.405 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /proc 2023-12-17T11:55:54.405 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /dev 2023-12-17T11:55:54.405 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /run 2023-12-17T11:55:54.405 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /mnt 2023-12-17T11:55:54.406 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /var/tmp 2023-12-17T11:55:54.406 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /home 2023-12-17T11:55:54.406 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /tmp 2023-12-17T11:55:54.406 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /dev 2023-12-17T11:55:54.406 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:55:54.801 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 108/162 2023-12-17T11:55:56.553 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 108/162 2023-12-17T11:55:56.760 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 109/162 2023-12-17T11:55:57.044 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 109/162 2023-12-17T11:55:57.184 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 110/162 2023-12-17T11:55:57.709 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 110/162 2023-12-17T11:55:57.883 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 111/162 2023-12-17T11:55:57.927 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 111/162 2023-12-17T11:56:00.784 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 112/162 2023-12-17T11:56:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 107/162 2023-12-17T11:56:00.926 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /run 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2023-12-17T11:56:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:56:00.937 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 112/162 2023-12-17T11:56:01.084 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 113/162 2023-12-17T11:56:01.320 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 108/162 2023-12-17T11:56:01.341 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 114/162 2023-12-17T11:56:01.566 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 115/162 2023-12-17T11:56:01.837 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 115/162 2023-12-17T11:56:02.025 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 116/162 2023-12-17T11:56:03.109 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 116/162 2023-12-17T11:56:03.172 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 108/162 2023-12-17T11:56:03.290 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 117/162 2023-12-17T11:56:03.393 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 107/162 2023-12-17T11:56:03.393 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2023-12-17T11:56:03.393 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2023-12-17T11:56:03.393 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /dev 2023-12-17T11:56:03.393 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /run 2023-12-17T11:56:03.394 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2023-12-17T11:56:03.394 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2023-12-17T11:56:03.394 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2023-12-17T11:56:03.394 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2023-12-17T11:56:03.394 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /dev 2023-12-17T11:56:03.394 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:56:03.423 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 117/162 2023-12-17T11:56:03.427 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 109/162 2023-12-17T11:56:03.729 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 109/162 2023-12-17T11:56:03.789 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 108/162 2023-12-17T11:56:03.855 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 118/162 2023-12-17T11:56:03.886 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 110/162 2023-12-17T11:56:04.034 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 119/162 2023-12-17T11:56:04.103 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 119/162 2023-12-17T11:56:04.266 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 120/162 2023-12-17T11:56:04.296 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 120/162 2023-12-17T11:56:04.447 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 110/162 2023-12-17T11:56:04.620 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 111/162 2023-12-17T11:56:04.664 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 111/162 2023-12-17T11:56:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 108/162 2023-12-17T11:56:05.723 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 121/162 2023-12-17T11:56:05.737 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 109/162 2023-12-17T11:56:05.933 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 122/162 2023-12-17T11:56:05.996 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 122/162 2023-12-17T11:56:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 109/162 2023-12-17T11:56:06.166 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 123/162 2023-12-17T11:56:06.214 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 110/162 2023-12-17T11:56:06.238 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 123/162 2023-12-17T11:56:06.399 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 124/162 2023-12-17T11:56:06.735 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 110/162 2023-12-17T11:56:06.817 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 124/162 2023-12-17T11:56:06.930 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 111/162 2023-12-17T11:56:06.972 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 111/162 2023-12-17T11:56:06.982 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 125/162 2023-12-17T11:56:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 112/162 2023-12-17T11:56:07.914 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 112/162 2023-12-17T11:56:08.087 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 113/162 2023-12-17T11:56:08.305 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 114/162 2023-12-17T11:56:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 115/162 2023-12-17T11:56:08.840 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 115/162 2023-12-17T11:56:09.026 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 116/162 2023-12-17T11:56:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 112/162 2023-12-17T11:56:10.156 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 112/162 2023-12-17T11:56:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 116/162 2023-12-17T11:56:10.297 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 113/162 2023-12-17T11:56:10.352 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 117/162 2023-12-17T11:56:10.493 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 117/162 2023-12-17T11:56:10.552 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 114/162 2023-12-17T11:56:10.829 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 115/162 2023-12-17T11:56:10.908 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 118/162 2023-12-17T11:56:11.094 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 119/162 2023-12-17T11:56:11.095 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 115/162 2023-12-17T11:56:11.166 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 119/162 2023-12-17T11:56:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 120/162 2023-12-17T11:56:11.335 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 116/162 2023-12-17T11:56:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 120/162 2023-12-17T11:56:12.025 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 125/162 2023-12-17T11:56:12.418 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 116/162 2023-12-17T11:56:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 117/162 2023-12-17T11:56:12.751 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 117/162 2023-12-17T11:56:12.861 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 121/162 2023-12-17T11:56:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 122/162 2023-12-17T11:56:13.168 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 122/162 2023-12-17T11:56:13.197 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 118/162 2023-12-17T11:56:13.357 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 126/162 2023-12-17T11:56:13.386 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 123/162 2023-12-17T11:56:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 119/162 2023-12-17T11:56:13.461 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 123/162 2023-12-17T11:56:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 119/162 2023-12-17T11:56:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 124/162 2023-12-17T11:56:13.677 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 127/162 2023-12-17T11:56:13.687 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 120/162 2023-12-17T11:56:13.717 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 120/162 2023-12-17T11:56:13.948 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2023-12-17T11:56:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 124/162 2023-12-17T11:56:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 125/162 2023-12-17T11:56:14.391 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 129/162 2023-12-17T11:56:14.728 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 130/162 2023-12-17T11:56:14.947 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2023-12-17T11:56:14.977 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2023-12-17T11:56:15.150 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 121/162 2023-12-17T11:56:15.244 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2023-12-17T11:56:15.360 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 122/162 2023-12-17T11:56:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 122/162 2023-12-17T11:56:15.435 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2023-12-17T11:56:15.598 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 134/162 2023-12-17T11:56:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 123/162 2023-12-17T11:56:15.709 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 123/162 2023-12-17T11:56:15.920 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 124/162 2023-12-17T11:56:16.340 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 124/162 2023-12-17T11:56:16.570 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 125/162 2023-12-17T11:56:19.020 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:19.335 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:19.423 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:19.663 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 125/162 2023-12-17T11:56:19.766 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2023-12-17T11:56:20.155 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2023-12-17T11:56:20.263 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 126/162 2023-12-17T11:56:20.434 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 138/162 2023-12-17T11:56:20.622 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 139/162 2023-12-17T11:56:20.627 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 127/162 2023-12-17T11:56:20.839 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 140/162 2023-12-17T11:56:20.867 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2023-12-17T11:56:21.126 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 141/162 2023-12-17T11:56:21.415 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 142/162 2023-12-17T11:56:21.617 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 125/162 2023-12-17T11:56:21.687 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2023-12-17T11:56:21.804 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2023-12-17T11:56:22.007 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2023-12-17T11:56:22.012 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2023-12-17T11:56:22.042 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2023-12-17T11:56:22.163 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 126/162 2023-12-17T11:56:22.212 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 145/162 2023-12-17T11:56:22.235 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 145/162 2023-12-17T11:56:22.331 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 129/162 2023-12-17T11:56:22.382 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2023-12-17T11:56:22.528 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 127/162 2023-12-17T11:56:22.568 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2023-12-17T11:56:22.600 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2023-12-17T11:56:22.710 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2023-12-17T11:56:22.739 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2023-12-17T11:56:22.850 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 131/162 2023-12-17T11:56:22.942 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2023-12-17T11:56:22.942 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2023-12-17T11:56:23.065 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2023-12-17T11:56:23.208 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2023-12-17T11:56:23.208 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2023-12-17T11:56:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2023-12-17T11:56:23.452 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2023-12-17T11:56:23.452 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2023-12-17T11:56:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 134/162 2023-12-17T11:56:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2023-12-17T11:56:23.731 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2023-12-17T11:56:23.964 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2023-12-17T11:56:24.057 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 129/162 2023-12-17T11:56:24.139 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2023-12-17T11:56:24.352 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2023-12-17T11:56:24.397 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2023-12-17T11:56:24.397 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2023-12-17T11:56:24.543 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 131/162 2023-12-17T11:56:24.657 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2023-12-17T11:56:24.657 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2023-12-17T11:56:24.774 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2023-12-17T11:56:24.915 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2023-12-17T11:56:24.915 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:24.999 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2023-12-17T11:56:25.170 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 134/162 2023-12-17T11:56:25.173 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:25.173 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2023-12-17T11:56:25.398 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2023-12-17T11:56:25.398 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:25.665 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:25.815 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2023-12-17T11:56:25.815 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2023-12-17T11:56:26.405 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2023-12-17T11:56:26.405 INFO:teuthology.orchestra.run.smithi141.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-17T11:56:26.405 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:56:27.106 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:27.134 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/162 2023-12-17T11:56:27.134 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/162 2023-12-17T11:56:27.134 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/162 2023-12-17T11:56:27.135 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/162 2023-12-17T11:56:27.136 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/162 2023-12-17T11:56:27.148 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/162 2023-12-17T11:56:27.149 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/162 2023-12-17T11:56:27.150 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2023-12-17T11:56:27.151 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 88/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 89/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 90/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 91/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 92/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 93/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 94/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/162 2023-12-17T11:56:27.152 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 96/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 97/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 98/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 99/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2023-12-17T11:56:27.153 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 111/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 112/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 114/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 115/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 116/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 117/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 118/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 119/162 2023-12-17T11:56:27.154 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 120/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 121/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 122/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : automake-1.16.1-8.el8.noarch 123/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libpq-13.11-1.el8.x86_64 124/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : jq-1.6-7.el8.x86_64 125/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 126/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 127/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 128/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 129/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 130/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 131/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2023-12-17T11:56:27.155 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 133/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 135/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 136/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 137/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 138/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 139/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 140/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 141/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 142/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 143/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 144/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 145/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 146/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 147/162 2023-12-17T11:56:27.156 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 148/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 149/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 150/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 151/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 152/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 153/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 154/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 155/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 156/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 157/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 159/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:27.157 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2023-12-17T11:56:27.541 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:27.609 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2023-12-17T11:56:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2023-12-17T11:56:28.647 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 138/162 2023-12-17T11:56:28.664 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 139/162 2023-12-17T11:56:29.067 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 140/162 2023-12-17T11:56:29.133 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 135/162 2023-12-17T11:56:29.438 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 141/162 2023-12-17T11:56:29.514 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2023-12-17T11:56:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 142/162 2023-12-17T11:56:29.932 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 143/162 2023-12-17T11:56:29.957 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2023-12-17T11:56:29.960 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 143/162 2023-12-17T11:56:30.159 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2023-12-17T11:56:30.240 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 138/162 2023-12-17T11:56:30.436 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2023-12-17T11:56:30.469 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 139/162 2023-12-17T11:56:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2023-12-17T11:56:30.736 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 140/162 2023-12-17T11:56:30.774 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2023-12-17T11:56:30.806 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2023-12-17T11:56:31.022 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2023-12-17T11:56:31.054 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 141/162 2023-12-17T11:56:31.193 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2023-12-17T11:56:31.353 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 142/162 2023-12-17T11:56:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2023-12-17T11:56:31.372 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2023-12-17T11:56:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 143/162 2023-12-17T11:56:31.595 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 143/162 2023-12-17T11:56:31.704 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2023-12-17T11:56:31.705 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2023-12-17T11:56:31.817 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2023-12-17T11:56:31.957 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2023-12-17T11:56:31.957 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2023-12-17T11:56:32.049 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2023-12-17T11:56:32.165 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2023-12-17T11:56:32.247 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2023-12-17T11:56:32.392 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2023-12-17T11:56:32.422 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2023-12-17T11:56:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2023-12-17T11:56:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2023-12-17T11:56:32.693 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2023-12-17T11:56:32.853 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2023-12-17T11:56:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2023-12-17T11:56:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2023-12-17T11:56:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2023-12-17T11:56:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2023-12-17T11:56:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2023-12-17T11:56:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2023-12-17T11:56:33.380 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2023-12-17T11:56:33.381 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2023-12-17T11:56:33.502 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2023-12-17T11:56:33.503 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:33.674 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2023-12-17T11:56:33.674 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2023-12-17T11:56:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2023-12-17T11:56:34.000 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2023-12-17T11:56:34.192 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2023-12-17T11:56:34.192 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2023-12-17T11:56:34.405 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2023-12-17T11:56:34.677 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2023-12-17T11:56:34.678 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2023-12-17T11:56:34.971 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2023-12-17T11:56:34.971 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2023-12-17T11:56:35.237 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2023-12-17T11:56:35.237 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:35.520 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:35.520 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2023-12-17T11:56:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:35.735 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2023-12-17T11:56:35.736 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2023-12-17T11:56:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2023-12-17T11:56:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:36.087 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:36.245 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2023-12-17T11:56:36.245 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2023-12-17T11:56:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2023-12-17T11:56:36.373 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-17T11:56:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:56:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2023-12-17T11:56:36.878 INFO:teuthology.orchestra.run.smithi192.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-17T11:56:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:56:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/162 2023-12-17T11:56:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/162 2023-12-17T11:56:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/162 2023-12-17T11:56:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/162 2023-12-17T11:56:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/162 2023-12-17T11:56:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/162 2023-12-17T11:56:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/162 2023-12-17T11:56:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/162 2023-12-17T11:56:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/162 2023-12-17T11:56:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/162 2023-12-17T11:56:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/162 2023-12-17T11:56:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2023-12-17T11:56:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 84/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 85/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 86/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 87/162 2023-12-17T11:56:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 88/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 89/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 90/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 91/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 92/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 93/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 94/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 95/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.1-8.el8.noarch 96/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpq-13.11-1.el8.x86_64 97/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : jq-1.6-7.el8.x86_64 98/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 99/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 100/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 101/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 102/162 2023-12-17T11:56:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 103/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2023-12-17T11:56:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 119/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 120/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 121/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 122/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 123/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 124/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 125/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 126/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 127/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 129/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 130/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 131/162 2023-12-17T11:56:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 133/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 135/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 136/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 137/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 138/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 139/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 140/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 141/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 142/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 143/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 144/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 145/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 146/162 2023-12-17T11:56:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 147/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 148/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 149/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 150/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 151/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 152/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 153/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 154/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 155/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 156/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 157/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 159/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2023-12-17T11:56:37.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/162 2023-12-17T11:56:37.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/162 2023-12-17T11:56:37.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/162 2023-12-17T11:56:37.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/162 2023-12-17T11:56:37.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/162 2023-12-17T11:56:37.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/162 2023-12-17T11:56:37.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/162 2023-12-17T11:56:37.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/162 2023-12-17T11:56:37.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/162 2023-12-17T11:56:37.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/162 2023-12-17T11:56:37.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/162 2023-12-17T11:56:37.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/162 2023-12-17T11:56:37.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/162 2023-12-17T11:56:37.695 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/162 2023-12-17T11:56:37.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/162 2023-12-17T11:56:37.697 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2023-12-17T11:56:37.698 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 84/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 85/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 86/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 87/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 88/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 89/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 90/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 91/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 92/162 2023-12-17T11:56:37.699 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 93/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 94/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 95/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.1-8.el8.noarch 96/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpq-13.11-1.el8.x86_64 97/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-7.el8.x86_64 98/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 99/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 100/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 101/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 102/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 103/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2023-12-17T11:56:37.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 119/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 120/162 2023-12-17T11:56:37.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 121/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 122/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 123/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 124/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 125/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 126/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 127/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 129/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 130/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 131/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 133/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2023-12-17T11:56:37.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 135/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 136/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 137/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 138/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 139/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 140/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 141/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 142/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 143/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 144/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 145/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 146/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 147/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 148/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 149/162 2023-12-17T11:56:37.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 150/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 151/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 152/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 153/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 154/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 155/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 156/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 157/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 158/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 159/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 160/162 2023-12-17T11:56:37.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2023-12-17T11:56:39.454 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2023-12-17T11:56:39.454 INFO:teuthology.orchestra.run.smithi141.stdout:Installed products updated. 2023-12-17T11:56:40.113 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:56:40.113 INFO:teuthology.orchestra.run.smithi141.stdout:Upgraded: 2023-12-17T11:56:40.113 INFO:teuthology.orchestra.run.smithi141.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-17T11:56:40.113 INFO:teuthology.orchestra.run.smithi141.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-17T11:56:40.113 INFO:teuthology.orchestra.run.smithi141.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-17T11:56:40.113 INFO:teuthology.orchestra.run.smithi141.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-17T11:56:40.113 INFO:teuthology.orchestra.run.smithi141.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout:Installed: 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-17T11:56:40.114 INFO:teuthology.orchestra.run.smithi141.stdout: automake-1.16.1-8.el8.noarch 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: bison-3.0.4-10.el8.x86_64 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.115 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: flex-2.6.1-9.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-17T11:56:40.116 INFO:teuthology.orchestra.run.smithi141.stdout: jq-1.6-7.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libpq-13.11-1.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.117 INFO:teuthology.orchestra.run.smithi141.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-17T11:56:40.118 INFO:teuthology.orchestra.run.smithi141.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.119 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-17T11:56:40.120 INFO:teuthology.orchestra.run.smithi141.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-portend-2.6-1.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-17T11:56:40.121 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-17T11:56:40.122 INFO:teuthology.orchestra.run.smithi141.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-17T11:56:40.123 INFO:teuthology.orchestra.run.smithi141.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-17T11:56:40.124 INFO:teuthology.orchestra.run.smithi141.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-17T11:56:40.124 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-12-17T11:56:40.124 INFO:teuthology.orchestra.run.smithi141.stdout:Complete! 2023-12-17T11:56:42.329 DEBUG:teuthology.parallel:result is None 2023-12-17T11:56:53.212 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2023-12-17T11:56:53.212 INFO:teuthology.orchestra.run.smithi027.stdout:Installed products updated. 2023-12-17T11:56:53.896 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:56:53.896 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2023-12-17T11:56:53.896 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-17T11:56:53.896 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-17T11:56:53.896 INFO:teuthology.orchestra.run.smithi027.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-17T11:56:53.897 INFO:teuthology.orchestra.run.smithi027.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-17T11:56:53.897 INFO:teuthology.orchestra.run.smithi027.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-17T11:56:53.897 INFO:teuthology.orchestra.run.smithi027.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-17T11:56:53.897 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.897 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.897 INFO:teuthology.orchestra.run.smithi027.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-17T11:56:53.898 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-17T11:56:53.898 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:53.898 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:53.898 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-17T11:56:53.898 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-17T11:56:53.898 INFO:teuthology.orchestra.run.smithi027.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.1-8.el8.noarch 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.0.4-10.el8.x86_64 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.899 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.900 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.1-9.el8.x86_64 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-17T11:56:53.901 INFO:teuthology.orchestra.run.smithi027.stdout: jq-1.6-7.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libpq-13.11-1.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.902 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-17T11:56:53.903 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.904 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-17T11:56:53.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-17T11:56:53.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-17T11:56:53.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-17T11:56:53.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-17T11:56:53.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-17T11:56:53.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-17T11:56:53.905 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-17T11:56:53.906 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-2.6-1.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-17T11:56:53.907 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-17T11:56:53.908 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.909 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.909 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:53.909 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-17T11:56:53.909 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-17T11:56:53.909 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-17T11:56:53.909 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-17T11:56:53.909 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-17T11:56:53.909 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-17T11:56:53.910 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T11:56:53.910 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-12-17T11:56:54.280 DEBUG:teuthology.parallel:result is None 2023-12-17T11:56:54.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2023-12-17T11:56:54.701 INFO:teuthology.orchestra.run.smithi192.stdout:Installed products updated. 2023-12-17T11:56:56.037 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:56:56.038 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2023-12-17T11:56:56.038 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-17T11:56:56.038 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-17T11:56:56.038 INFO:teuthology.orchestra.run.smithi192.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-17T11:56:56.038 INFO:teuthology.orchestra.run.smithi192.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-17T11:56:56.039 INFO:teuthology.orchestra.run.smithi192.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-17T11:56:56.039 INFO:teuthology.orchestra.run.smithi192.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-17T11:56:56.039 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.039 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.039 INFO:teuthology.orchestra.run.smithi192.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-17T11:56:56.039 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-17T11:56:56.039 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:56.039 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.1-8.el8.noarch 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.0.4-10.el8.x86_64 2023-12-17T11:56:56.040 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.042 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.043 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.043 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.043 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.043 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-17T11:56:56.043 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-17T11:56:56.043 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.1-9.el8.x86_64 2023-12-17T11:56:56.043 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: jq-1.6-7.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-17T11:56:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: libpq-13.11-1.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-17T11:56:56.045 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-17T11:56:56.046 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-17T11:56:56.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-17T11:56:56.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-2.6-1.el8.noarch 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-17T11:56:56.049 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.050 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-17T11:56:56.051 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-17T11:56:56.052 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.052 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.052 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-17T11:56:56.052 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-17T11:56:56.052 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-17T11:56:56.052 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-17T11:56:56.052 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-17T11:56:56.052 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-17T11:56:56.053 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-17T11:56:56.053 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T11:56:56.053 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-12-17T11:56:56.411 DEBUG:teuthology.parallel:result is None 2023-12-17T11:56:56.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:56:56.559 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-17T11:56:56.621 INFO:teuthology.orchestra.run.smithi027.stdout:19.0.0-119.ga62fda73.el8 2023-12-17T11:56:56.622 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-17T11:56:56.622 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-17T11:56:56.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:56:56.764 DEBUG:teuthology.orchestra.run.smithi141:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-17T11:56:56.826 INFO:teuthology.orchestra.run.smithi141.stdout:19.0.0-119.ga62fda73.el8 2023-12-17T11:56:56.827 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-17T11:56:56.827 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-17T11:56:56.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:56:57.077 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-17T11:56:57.135 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-119.ga62fda73.el8 2023-12-17T11:56:57.136 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-17T11:56:57.136 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-17T11:56:57.137 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-17T11:56:57.138 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T11:56:57.138 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-17T11:56:57.237 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T11:56:57.237 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-17T11:56:57.333 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T11:56:57.333 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-17T11:56:57.428 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-17T11:56:57.428 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T11:56:57.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2023-12-17T11:56:57.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-17T11:56:57.614 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T11:56:57.614 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/usr/bin/daemon-helper 2023-12-17T11:56:57.689 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-17T11:56:57.799 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T11:56:57.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2023-12-17T11:56:57.873 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-17T11:56:57.982 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-17T11:56:57.982 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T11:56:57.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-17T11:56:58.060 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-17T11:56:58.172 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T11:56:58.173 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-17T11:56:58.243 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-17T11:56:58.351 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T11:56:58.351 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-17T11:56:58.420 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-17T11:56:58.528 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-17T11:56:58.528 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T11:56:58.528 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2023-12-17T11:56:58.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-17T11:56:58.713 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T11:56:58.713 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/usr/bin/stdin-killer 2023-12-17T11:56:58.781 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-17T11:56:58.888 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T11:56:58.888 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2023-12-17T11:56:58.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-17T11:56:59.065 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-17T11:56:59.145 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'subvols': {'create': 2}} 2023-12-17T11:56:59.145 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:56:59.146 INFO:tasks.cephadm:Cluster fsid is 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T11:56:59.146 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-17T11:56:59.146 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.27', 'mon.b': '172.21.15.141', 'mon.c': '172.21.15.192'} 2023-12-17T11:56:59.146 INFO:tasks.cephadm:First mon is mon.a on smithi027 2023-12-17T11:56:59.146 INFO:tasks.cephadm:First mgr is x 2023-12-17T11:56:59.146 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-17T11:56:59.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2023-12-17T11:56:59.223 DEBUG:teuthology.orchestra.run.smithi141:> sudo hostname $(hostname -s) 2023-12-17T11:56:59.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2023-12-17T11:56:59.365 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-17T11:56:59.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:56:59.506 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74860/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-119-ga62fda73', 'node_name': '172.21.4.63+confusa01', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-119.ga62fda73'}, 'url': 'https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-11 13:31:01.711792', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2023dec11', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74860/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-119-ga62fda73', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-119.ga62fda73'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-11 12:43:31.974707', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2023dec11', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-12-17T11:56:59.719 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref rishabh-2023dec11, sha1 a62fda73efba5520d409bb5a21bd8373289dda5d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-17T11:56:59.721 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm 2023-12-17T11:56:59.721 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:00.400 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 17 11:57 /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:00.400 DEBUG:teuthology.orchestra.run.smithi141:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:01.224 INFO:teuthology.orchestra.run.smithi141.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 17 11:57 /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:01.225 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:01.847 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 17 11:57 /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:01.848 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:01.901 DEBUG:teuthology.orchestra.run.smithi141:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:01.951 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-17T11:57:02.019 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d on all hosts... 2023-12-17T11:57:02.020 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-17T11:57:02.025 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-17T11:57:02.029 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-17T11:57:03.568 INFO:teuthology.orchestra.run.smithi141.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-17T11:57:03.646 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-17T11:57:03.667 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-17T12:00:00.708 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-17T12:00:00.708 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-17T12:00:00.708 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-17T12:00:00.708 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2023-12-17T12:00:00.709 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-17T12:00:00.709 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-17T12:00:00.709 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-17T12:01:15.611 INFO:teuthology.orchestra.run.smithi141.stdout:{ 2023-12-17T12:01:15.611 INFO:teuthology.orchestra.run.smithi141.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-17T12:01:15.612 INFO:teuthology.orchestra.run.smithi141.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-17T12:01:15.612 INFO:teuthology.orchestra.run.smithi141.stdout: "repo_digests": [ 2023-12-17T12:01:15.612 INFO:teuthology.orchestra.run.smithi141.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-17T12:01:15.612 INFO:teuthology.orchestra.run.smithi141.stdout: ] 2023-12-17T12:01:15.612 INFO:teuthology.orchestra.run.smithi141.stdout:} 2023-12-17T12:01:16.574 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-17T12:01:16.574 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-17T12:01:16.574 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-17T12:01:16.574 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2023-12-17T12:01:16.574 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-17T12:01:16.575 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2023-12-17T12:01:16.575 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-17T12:01:16.608 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2023-12-17T12:01:16.693 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /etc/ceph 2023-12-17T12:01:16.773 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2023-12-17T12:01:16.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2023-12-17T12:01:16.920 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod 777 /etc/ceph 2023-12-17T12:01:16.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2023-12-17T12:01:17.058 INFO:tasks.cephadm:Writing seed config... 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-17T12:01:17.060 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-17T12:01:17.061 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-17T12:01:17.062 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-12-17T12:01:17.063 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:01:17.063 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-17T12:01:17.109 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 62030760-9cd3-11ee-95a5-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-17T12:01:17.110 DEBUG:teuthology.orchestra.run.smithi027:mon.a> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.a.service 2023-12-17T12:01:17.155 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.x.service 2023-12-17T12:01:17.200 INFO:tasks.cephadm:Bootstrapping... 2023-12-17T12:01:17.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d -v bootstrap --fsid 62030760-9cd3-11ee-95a5-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.27 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:01:17.227 INFO:journalctl@ceph.mon.a.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:01:17.271 INFO:journalctl@ceph.mgr.x.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:01:17.487 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-12-17T12:01:17.487 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d', '-v', 'bootstrap', '--fsid', '62030760-9cd3-11ee-95a5-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.27', '--skip-admin-label'] 2023-12-17T12:01:17.536 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 4.6.1 2023-12-17T12:01:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-17T12:01:17.537 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying podman|docker is present... 2023-12-17T12:01:17.583 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 4.6.1 2023-12-17T12:01:17.583 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying lvm2 is present... 2023-12-17T12:01:17.583 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying time synchronization is in place... 2023-12-17T12:01:17.599 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-17T12:01:17.600 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-17T12:01:17.613 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-17T12:01:17.613 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2023-12-17T12:01:17.627 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2023-12-17T12:01:17.641 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2023-12-17T12:01:17.641 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2023-12-17T12:01:17.641 INFO:teuthology.orchestra.run.smithi027.stdout:Repeating the final host check... 2023-12-17T12:01:17.687 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 4.6.1 2023-12-17T12:01:17.687 INFO:teuthology.orchestra.run.smithi027.stdout:podman (/bin/podman) version 4.6.1 is present 2023-12-17T12:01:17.687 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl is present 2023-12-17T12:01:17.687 INFO:teuthology.orchestra.run.smithi027.stdout:lvcreate is present 2023-12-17T12:01:17.701 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-17T12:01:17.701 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-17T12:01:17.715 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-17T12:01:17.715 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2023-12-17T12:01:17.729 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2023-12-17T12:01:17.742 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2023-12-17T12:01:17.743 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2023-12-17T12:01:17.743 INFO:teuthology.orchestra.run.smithi027.stdout:Host looks OK 2023-12-17T12:01:17.743 INFO:teuthology.orchestra.run.smithi027.stdout:Cluster fsid: 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:17.743 INFO:teuthology.orchestra.run.smithi027.stdout:Acquiring lock 140209490168688 on /run/cephadm/62030760-9cd3-11ee-95a5-87774f69a715.lock 2023-12-17T12:01:17.743 INFO:teuthology.orchestra.run.smithi027.stdout:Lock 140209490168688 acquired on /run/cephadm/62030760-9cd3-11ee-95a5-87774f69a715.lock 2023-12-17T12:01:17.743 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 3300 ... 2023-12-17T12:01:17.744 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 6789 ... 2023-12-17T12:01:17.744 INFO:teuthology.orchestra.run.smithi027.stdout:Base mon IP(s) is [172.21.15.27:3300, 172.21.15.27:6789], mon addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2023-12-17T12:01:17.753 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-12-17T12:01:17.753 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2023-12-17T12:01:17.761 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-17T12:01:17.761 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-12-17T12:01:17.761 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2023-12-17T12:01:17.768 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-17T12:01:17.768 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-17T12:01:17.768 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-17T12:01:17.768 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-12-17T12:01:17.768 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-12-17T12:01:17.768 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-17T12:01:17.769 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2023-12-17T12:01:17.769 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2023-12-17T12:01:17.769 INFO:teuthology.orchestra.run.smithi027.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-12-17T12:01:17.770 INFO:teuthology.orchestra.run.smithi027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-17T12:01:17.771 INFO:teuthology.orchestra.run.smithi027.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-17T12:01:18.196 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a 2023-12-17T12:01:18.196 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-17T12:01:18.196 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Getting image source signatures 2023-12-17T12:01:18.197 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:d3e436ac93e4b5a6d94c7ac8024df331e66e67275836276615dc044fd3cbf83a 2023-12-17T12:01:18.197 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:933954f0af8f706d6a6186e87e0ea462c846b5abc6ef29ee9ef20dac46eff0f7 2023-12-17T12:01:18.197 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying config sha256:0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a 2023-12-17T12:01:18.197 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Writing manifest to image destination 2023-12-17T12:01:18.938 INFO:teuthology.orchestra.run.smithi027.stdout:ceph: stdout ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev) 2023-12-17T12:01:18.938 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph version: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev) 2023-12-17T12:01:18.938 INFO:teuthology.orchestra.run.smithi027.stdout:Extracting ceph user uid/gid from container image... 2023-12-17T12:01:19.527 INFO:teuthology.orchestra.run.smithi027.stdout:stat: stdout 167 167 2023-12-17T12:01:19.527 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial keys... 2023-12-17T12:01:20.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQAP435l+OU+MBAAnegXjtB5eue1yXiLuvmMrA== 2023-12-17T12:01:20.768 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQAQ435loqrXFxAA/6K8sAUhO0CY8/F5xZN3QQ== 2023-12-17T12:01:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQAR435lKHxwBBAAHVKbKa/Elw/rrrRhryMdpg== 2023-12-17T12:01:21.402 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial monmap... 2023-12-17T12:01:22.027 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-17T12:01:22.027 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-17T12:01:22.027 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:22.027 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-17T12:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool for a [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-17T12:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = pacific 2023-12-17T12:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: set fsid to 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-17T12:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mon... 2023-12-17T12:01:22.756 INFO:teuthology.orchestra.run.smithi027.stdout:create mon.a on 2023-12-17T12:01:23.011 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-17T12:01:23.221 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-17T12:01:23.484 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-62030760-9cd3-11ee-95a5-87774f69a715.target → /etc/systemd/system/ceph-62030760-9cd3-11ee-95a5-87774f69a715.target. 2023-12-17T12:01:23.484 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-62030760-9cd3-11ee-95a5-87774f69a715.target → /etc/systemd/system/ceph-62030760-9cd3-11ee-95a5-87774f69a715.target. 2023-12-17T12:01:23.862 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.a 2023-12-17T12:01:23.862 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.a.service: Unit ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.a.service not loaded. 2023-12-17T12:01:24.078 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-62030760-9cd3-11ee-95a5-87774f69a715.target.wants/ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.a.service → /etc/systemd/system/ceph-62030760-9cd3-11ee-95a5-87774f69a715@.service. 2023-12-17T12:01:24.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:24 smithi027 systemd[1]: Starting Ceph mon.a for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:01:24.579 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-17T12:01:24.580 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2023-12-17T12:01:24.597 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-17T12:01:24.597 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2023-12-17T12:01:24.598 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2023-12-17T12:01:24.598 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2023-12-17T12:01:24.599 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon to start... 2023-12-17T12:01:24.599 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon... 2023-12-17T12:01:24.813 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:24 smithi027 bash[36348]: c2692364fed2ed227569044d90d6d7945955ab370b9530b85202ba0d6bf1ce40 2023-12-17T12:01:24.813 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:24 smithi027 systemd[1]: Started Ceph mon.a for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:01:24.813 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:24 smithi027 ceph-mon[36386]: mkfs 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:24.813 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:24 smithi027 ceph-mon[36386]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-17T12:01:25.574 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout cluster: 2023-12-17T12:01:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout id: 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-17T12:01:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout services: 2023-12-17T12:01:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.592405s) 2023-12-17T12:01:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-17T12:01:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout data: 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pgs: 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.084+0000 7f27d1d6a700 1 Processor -- start 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.084+0000 7f27d1d6a700 1 -- start start 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.085+0000 7f27d1d6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc07b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.085+0000 7f27d1d6a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc07b860 con 0x7f27cc07ce30 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.085+0000 7f27cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc07b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.085+0000 7f27cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc07b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35994/0 (socket says 172.21.15.27:35994) 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.085+0000 7f27cb7fe700 1 -- 172.21.15.27:0/899048889 learned_addr learned my addr 172.21.15.27:0/899048889 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.086+0000 7f27cb7fe700 1 -- 172.21.15.27:0/899048889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc07b9a0 con 0x7f27cc07ce30 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.086+0000 7f27cb7fe700 1 --2- 172.21.15.27:0/899048889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc07b290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0098a0 tx=0x7f27bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09827f5e8723809 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.086+0000 7f27ca7fc700 1 -- 172.21.15.27:0/899048889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27bc004160 con 0x7f27cc07ce30 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.086+0000 7f27ca7fc700 1 -- 172.21.15.27:0/899048889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc0042c0 con 0x7f27cc07ce30 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.086+0000 7f27ca7fc700 1 -- 172.21.15.27:0/899048889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27bc0045a0 con 0x7f27cc07ce30 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.087+0000 7f27d1d6a700 1 -- 172.21.15.27:0/899048889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 msgr2=0x7f27cc07b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.087+0000 7f27d1d6a700 1 --2- 172.21.15.27:0/899048889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc07b290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0098a0 tx=0x7f27bc009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.087+0000 7f27d1d6a700 1 -- 172.21.15.27:0/899048889 shutdown_connections 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.087+0000 7f27d1d6a700 1 --2- 172.21.15.27:0/899048889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc07b290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.087+0000 7f27d1d6a700 1 -- 172.21.15.27:0/899048889 >> 172.21.15.27:0/899048889 conn(0x7f27cc102140 msgr2=0x7f27cc104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.087+0000 7f27d1d6a700 1 -- 172.21.15.27:0/899048889 shutdown_connections 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.087+0000 7f27d1d6a700 1 -- 172.21.15.27:0/899048889 wait complete. 2023-12-17T12:01:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.088+0000 7f27d1d6a700 1 Processor -- start 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.088+0000 7f27d1d6a700 1 -- start start 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.089+0000 7f27d1d6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.089+0000 7f27d1d6a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc19c780 con 0x7f27cc07ce30 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.089+0000 7f27cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.089+0000 7f27cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc19c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35996/0 (socket says 172.21.15.27:35996) 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.089+0000 7f27cb7fe700 1 -- 172.21.15.27:0/327908747 learned_addr learned my addr 172.21.15.27:0/327908747 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.089+0000 7f27cb7fe700 1 -- 172.21.15.27:0/327908747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc009550 con 0x7f27cc07ce30 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.089+0000 7f27cb7fe700 1 --2- 172.21.15.27:0/327908747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc19c240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f27cc07c300 tx=0x7f27bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.090+0000 7f27c8ff9700 1 -- 172.21.15.27:0/327908747 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27bc011630 con 0x7f27cc07ce30 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.090+0000 7f27c8ff9700 1 -- 172.21.15.27:0/327908747 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc011790 con 0x7f27cc07ce30 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.090+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc19c980 con 0x7f27cc07ce30 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.090+0000 7f27c8ff9700 1 -- 172.21.15.27:0/327908747 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27bc011a30 con 0x7f27cc07ce30 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.090+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc19ce20 con 0x7f27cc07ce30 2023-12-17T12:01:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.090+0000 7f27c8ff9700 1 -- 172.21.15.27:0/327908747 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f27bc022070 con 0x7f27cc07ce30 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.090+0000 7f27c8ff9700 1 -- 172.21.15.27:0/327908747 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f27bc01c450 con 0x7f27cc07ce30 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.091+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc196120 con 0x7f27cc07ce30 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.094+0000 7f27c8ff9700 1 -- 172.21.15.27:0/327908747 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f27bc01b450 con 0x7f27cc07ce30 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.186+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f27cc07bfc0 con 0x7f27cc07ce30 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.186+0000 7f27c8ff9700 1 -- 172.21.15.27:0/327908747 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f27bc01b5f0 con 0x7f27cc07ce30 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.187+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 msgr2=0x7f27cc19c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.187+0000 7f27d1d6a700 1 --2- 172.21.15.27:0/327908747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc19c240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f27cc07c300 tx=0x7f27bc004750 comp rx=0 tx=0).stop 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.187+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 shutdown_connections 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.187+0000 7f27d1d6a700 1 --2- 172.21.15.27:0/327908747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27cc07ce30 0x7f27cc19c240 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.187+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 >> 172.21.15.27:0/327908747 conn(0x7f27cc102140 msgr2=0x7f27cc1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.187+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 shutdown_connections 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:25.187+0000 7f27d1d6a700 1 -- 172.21.15.27:0/327908747 wait complete. 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:mon is available 2023-12-17T12:01:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:Assimilating anything we can from ceph.conf... 2023-12-17T12:01:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:25 smithi027 ceph-mon[36386]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-17T12:01:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:25 smithi027 ceph-mon[36386]: monmap e1: 1 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:01:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:25 smithi027 ceph-mon[36386]: fsmap 2023-12-17T12:01:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:25 smithi027 ceph-mon[36386]: osdmap e1: 0 total, 0 up, 0 in 2023-12-17T12:01:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:25 smithi027 ceph-mon[36386]: mgrmap e1: no daemons active 2023-12-17T12:01:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:25 smithi027 ceph-mon[36386]: from='client.? 172.21.15.27:0/327908747' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-17T12:01:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-17T12:01:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.017+0000 7f3e8725e700 1 Processor -- start 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.017+0000 7f3e8725e700 1 -- start start 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.017+0000 7f3e8725e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e80105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.017+0000 7f3e8725e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80106450 con 0x7f3e80105a60 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.018+0000 7f3e84ffa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e80105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.018+0000 7f3e84ffa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e80105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36012/0 (socket says 172.21.15.27:36012) 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.018+0000 7f3e84ffa700 1 -- 172.21.15.27:0/3009317758 learned_addr learned my addr 172.21.15.27:0/3009317758 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.018+0000 7f3e84ffa700 1 -- 172.21.15.27:0/3009317758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80106c60 con 0x7f3e80105a60 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e84ffa700 1 --2- 172.21.15.27:0/3009317758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e80105e80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098a0 tx=0x7f3e74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99e238f53ec25b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e7f7fe700 1 -- 172.21.15.27:0/3009317758 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74004160 con 0x7f3e80105a60 2023-12-17T12:01:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e7f7fe700 1 -- 172.21.15.27:0/3009317758 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e740042c0 con 0x7f3e80105a60 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e7f7fe700 1 -- 172.21.15.27:0/3009317758 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e740045a0 con 0x7f3e80105a60 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e8725e700 1 -- 172.21.15.27:0/3009317758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 msgr2=0x7f3e80105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e8725e700 1 --2- 172.21.15.27:0/3009317758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e80105e80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098a0 tx=0x7f3e74009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e8725e700 1 -- 172.21.15.27:0/3009317758 shutdown_connections 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e8725e700 1 --2- 172.21.15.27:0/3009317758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e80105e80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.019+0000 7f3e8725e700 1 -- 172.21.15.27:0/3009317758 >> 172.21.15.27:0/3009317758 conn(0x7f3e80101020 msgr2=0x7f3e80103440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.020+0000 7f3e8725e700 1 -- 172.21.15.27:0/3009317758 shutdown_connections 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.020+0000 7f3e8725e700 1 -- 172.21.15.27:0/3009317758 wait complete. 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.021+0000 7f3e8725e700 1 Processor -- start 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.021+0000 7f3e8725e700 1 -- start start 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.021+0000 7f3e8725e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e8018e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.021+0000 7f3e8725e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8018eb10 con 0x7f3e80105a60 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.021+0000 7f3e84ffa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e8018e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.021+0000 7f3e84ffa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e8018e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36024/0 (socket says 172.21.15.27:36024) 2023-12-17T12:01:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.021+0000 7f3e84ffa700 1 -- 172.21.15.27:0/3922896621 learned_addr learned my addr 172.21.15.27:0/3922896621 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.022+0000 7f3e84ffa700 1 -- 172.21.15.27:0/3922896621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74009550 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.022+0000 7f3e84ffa700 1 --2- 172.21.15.27:0/3922896621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e8018e5d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3e74004000 tx=0x7f3e74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.022+0000 7f3e7dffb700 1 -- 172.21.15.27:0/3922896621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74011630 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.022+0000 7f3e7dffb700 1 -- 172.21.15.27:0/3922896621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74011790 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.022+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8018cd50 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.022+0000 7f3e7dffb700 1 -- 172.21.15.27:0/3922896621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74011a30 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.022+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8018d1f0 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.023+0000 7f3e7dffb700 1 -- 172.21.15.27:0/3922896621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3e74021070 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.023+0000 7f3e7dffb700 1 -- 172.21.15.27:0/3922896621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3e7401b440 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.024+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e80062c00 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.026+0000 7f3e7dffb700 1 -- 172.21.15.27:0/3922896621 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f3e7401a460 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.101+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3e80106fd0 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.107+0000 7f3e7dffb700 1 -- 172.21.15.27:0/3922896621 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f3e7401a600 con 0x7f3e80105a60 2023-12-17T12:01:26.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.108+0000 7f3e7dffb700 1 -- 172.21.15.27:0/3922896621 <== mon.0 v2:172.21.15.27:3300/0 8 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3e7401a7a0 con 0x7f3e80105a60 2023-12-17T12:01:26.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.109+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 msgr2=0x7f3e8018e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:26.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.109+0000 7f3e8725e700 1 --2- 172.21.15.27:0/3922896621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e8018e5d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3e74004000 tx=0x7f3e74004750 comp rx=0 tx=0).stop 2023-12-17T12:01:26.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.109+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 shutdown_connections 2023-12-17T12:01:26.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.109+0000 7f3e8725e700 1 --2- 172.21.15.27:0/3922896621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e80105a60 0x7f3e8018e5d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:26.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.109+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 >> 172.21.15.27:0/3922896621 conn(0x7f3e80101020 msgr2=0x7f3e80190e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:26.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.109+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 shutdown_connections 2023-12-17T12:01:26.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.109+0000 7f3e8725e700 1 -- 172.21.15.27:0/3922896621 wait complete. 2023-12-17T12:01:26.468 INFO:teuthology.orchestra.run.smithi027.stdout:Generating new minimal ceph.conf... 2023-12-17T12:01:26.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:26 smithi027 ceph-mon[36386]: from='client.? 172.21.15.27:0/3922896621' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-17T12:01:26.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:26 smithi027 ceph-mon[36386]: from='client.? 172.21.15.27:0/3922896621' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-17T12:01:27.279 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.865+0000 7f3a12705700 1 Processor -- start 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.866+0000 7f3a12705700 1 -- start start 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.866+0000 7f3a12705700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c07b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.866+0000 7f3a12705700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c07b880 con 0x7f3a0c07ce50 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.866+0000 7f3a0bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c07b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.866+0000 7f3a0bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c07b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40920/0 (socket says 172.21.15.27:40920) 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.866+0000 7f3a0bfff700 1 -- 172.21.15.27:0/947881341 learned_addr learned my addr 172.21.15.27:0/947881341 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.867+0000 7f3a0bfff700 1 -- 172.21.15.27:0/947881341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c07b9c0 con 0x7f3a0c07ce50 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.867+0000 7f3a0bfff700 1 --2- 172.21.15.27:0/947881341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c07b2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0098a0 tx=0x7f39fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff3c4e2fa39823b server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.867+0000 7f3a0affd700 1 -- 172.21.15.27:0/947881341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39fc004160 con 0x7f3a0c07ce50 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.867+0000 7f3a0affd700 1 -- 172.21.15.27:0/947881341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f39fc0042c0 con 0x7f3a0c07ce50 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.867+0000 7f3a0affd700 1 -- 172.21.15.27:0/947881341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39fc011940 con 0x7f3a0c07ce50 2023-12-17T12:01:27.280 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.868+0000 7f3a12705700 1 -- 172.21.15.27:0/947881341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 msgr2=0x7f3a0c07b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.868+0000 7f3a12705700 1 --2- 172.21.15.27:0/947881341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c07b2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0098a0 tx=0x7f39fc009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.868+0000 7f3a12705700 1 -- 172.21.15.27:0/947881341 shutdown_connections 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.868+0000 7f3a12705700 1 --2- 172.21.15.27:0/947881341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c07b2b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.868+0000 7f3a12705700 1 -- 172.21.15.27:0/947881341 >> 172.21.15.27:0/947881341 conn(0x7f3a0c102140 msgr2=0x7f3a0c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.868+0000 7f3a12705700 1 -- 172.21.15.27:0/947881341 shutdown_connections 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.868+0000 7f3a12705700 1 -- 172.21.15.27:0/947881341 wait complete. 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.869+0000 7f3a12705700 1 Processor -- start 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.869+0000 7f3a12705700 1 -- start start 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.870+0000 7f3a12705700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.870+0000 7f3a12705700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c19cf70 con 0x7f3a0c07ce50 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.870+0000 7f3a0bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:27.281 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.870+0000 7f3a0bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40936/0 (socket says 172.21.15.27:40936) 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.870+0000 7f3a0bfff700 1 -- 172.21.15.27:0/1197768712 learned_addr learned my addr 172.21.15.27:0/1197768712 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.870+0000 7f3a0bfff700 1 -- 172.21.15.27:0/1197768712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc009550 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.870+0000 7f3a0bfff700 1 --2- 172.21.15.27:0/1197768712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c19ca30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f39fc003f20 tx=0x7f39fc011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.871+0000 7f3a097fa700 1 -- 172.21.15.27:0/1197768712 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39fc0036a0 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.871+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c19d170 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.871+0000 7f3a097fa700 1 -- 172.21.15.27:0/1197768712 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f39fc01b440 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.871+0000 7f3a097fa700 1 -- 172.21.15.27:0/1197768712 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39fc023400 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.871+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c19d610 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.871+0000 7f3a097fa700 1 -- 172.21.15.27:0/1197768712 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f39fc023970 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.871+0000 7f3a097fa700 1 -- 172.21.15.27:0/1197768712 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f39fc02cdf0 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.872+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c10ceb0 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.875+0000 7f3a097fa700 1 -- 172.21.15.27:0/1197768712 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f39fc023d60 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.949+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3a0c10d070 con 0x7f3a0c07ce50 2023-12-17T12:01:27.282 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.950+0000 7f3a097fa700 1 -- 172.21.15.27:0/1197768712 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f39fc01b5b0 con 0x7f3a0c07ce50 2023-12-17T12:01:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.950+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 msgr2=0x7f3a0c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.950+0000 7f3a12705700 1 --2- 172.21.15.27:0/1197768712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c19ca30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f39fc003f20 tx=0x7f39fc011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.951+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 shutdown_connections 2023-12-17T12:01:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.951+0000 7f3a12705700 1 --2- 172.21.15.27:0/1197768712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c07ce50 0x7f3a0c19ca30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.951+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 >> 172.21.15.27:0/1197768712 conn(0x7f3a0c102140 msgr2=0x7f3a0c102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.951+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 shutdown_connections 2023-12-17T12:01:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:26.951+0000 7f3a12705700 1 -- 172.21.15.27:0/1197768712 wait complete. 2023-12-17T12:01:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:Restarting the monitor... 2023-12-17T12:01:27.593 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:27 smithi027 systemd[1]: Stopping Ceph mon.a for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:01:27.845 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:27 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-a[36363]: 2023-12-17T12:01:27.590+0000 7f05f4f23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:01:27.845 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:27 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-a[36363]: 2023-12-17T12:01:27.590+0000 7f05f4f23700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-17T12:01:27.845 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:27 smithi027 bash[36715]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-a 2023-12-17T12:01:28.654 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 systemd[1]: ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.a.service: Succeeded. 2023-12-17T12:01:28.654 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 systemd[1]: Stopped Ceph mon.a for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:01:28.654 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 systemd[1]: Starting Ceph mon.a for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:01:28.758 INFO:teuthology.orchestra.run.smithi027.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-12-17T12:01:29.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: set uid:gid to 167:167 (ceph:ceph) 2023-12-17T12:01:29.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev), process ceph-mon, pid 7 2023-12-17T12:01:29.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: pidfile_write: ignore empty --pid-file 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: load: jerasure load: lrc 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: RocksDB version: 7.9.2 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Git sha 0 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Compile date 2023-12-11 11:29:13 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: DB SUMMARY 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: DB Session ID: PGD7JN2LI7LV1K97WGXA 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: CURRENT file: CURRENT 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: IDENTITY file: IDENTITY 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 96966 ; 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.error_if_exists: 0 2023-12-17T12:01:29.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.create_if_missing: 0 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.paranoid_checks: 1 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.env: 0x55f881c3ea60 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.fs: PosixFileSystem 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.info_log: 0x55f883a9f780 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_file_opening_threads: 16 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.statistics: (nil) 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.use_fsync: 0 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_log_file_size: 0 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-17T12:01:29.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.keep_log_file_num: 1000 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.recycle_log_file_num: 0 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.allow_fallocate: 1 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.allow_mmap_reads: 0 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.allow_mmap_writes: 0 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.use_direct_reads: 0 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 bash[36834]: db878f6c8f5239d5ceb5c4840232c0df6ce8f1d7cce1509d631c203ebc6da762 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.create_missing_column_families: 0 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.db_log_dir: 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.wal_dir: 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-17T12:01:29.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.advise_random_on_open: 1 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.db_write_buffer_size: 0 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.write_buffer_manager: 0x55f883a55360 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.rate_limiter: (nil) 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.wal_recovery_mode: 2 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.enable_thread_tracking: 0 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.enable_pipelined_write: 0 2023-12-17T12:01:29.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.unordered_write: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.row_cache: None 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.wal_filter: None 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.allow_ingest_behind: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.two_write_queues: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.manual_wal_flush: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.wal_compression: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.atomic_flush: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-17T12:01:29.169 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.log_readahead_size: 0 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.best_efforts_recovery: 0 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.allow_data_in_errors: 0 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.db_host_id: __hostname__ 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_background_jobs: 2 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_background_compactions: -1 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_subcompactions: 1 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-17T12:01:29.170 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_total_wal_size: 0 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_open_files: -1 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bytes_per_sync: 0 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_readahead_size: 0 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_background_flushes: -1 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Compression algorithms supported: 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: kZSTD supported: 0 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: kXpressCompression supported: 0 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: kLZ4HCCompression supported: 1 2023-12-17T12:01:29.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: kZlibCompression supported: 1 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: kSnappyCompression supported: 1 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: kLZ4Compression supported: 1 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: kBZip2Compression supported: 0 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.merge_operator: 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_filter: None 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_filter_factory: None 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.sst_partitioner_factory: None 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f883a9f8a0) 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks: 1 2023-12-17T12:01:29.172 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_top_level_index_and_filter: 1 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_type: 0 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_index_type: 0 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_shortening: 1 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: checksum: 4 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: no_block_cache: 0 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache: 0x55f883af51f0 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_name: BinnedLRUCache 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_options: 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: capacity : 536870912 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_shard_bits : 4 2023-12-17T12:01:29.173 INFO:journalctl@ceph.mon.a.smithi027.stdout: strict_capacity_limit : 0 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: high_pri_pool_ratio: 0.000 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_compressed: (nil) 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: persistent_cache: (nil) 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size: 4096 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size_deviation: 10 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_restart_interval: 16 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_block_restart_interval: 1 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: metadata_block_size: 4096 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: partition_filters: 0 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: use_delta_encoding: 1 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: filter_policy: bloomfilter 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: whole_key_filtering: 1 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: verify_compression: 0 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: read_amp_bytes_per_bit: 0 2023-12-17T12:01:29.174 INFO:journalctl@ceph.mon.a.smithi027.stdout: format_version: 5 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_index_compression: 1 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_align: 0 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout: max_auto_readahead_size: 262144 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout: prepopulate_block_cache: 0 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout: initial_auto_readahead_size: 8192 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.write_buffer_size: 33554432 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_write_buffer_number: 2 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression: NoCompression 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression: Disabled 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.prefix_extractor: nullptr 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.num_levels: 7 2023-12-17T12:01:29.175 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-17T12:01:29.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-17T12:01:29.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-17T12:01:29.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-17T12:01:29.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-17T12:01:29.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-17T12:01:29.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.level: 32767 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.strategy: 0 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-17T12:01:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.enabled: false 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.target_file_size_base: 67108864 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-17T12:01:29.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.arena_block_size: 1048576 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.disable_auto_compactions: 0 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-17T12:01:29.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.table_properties_collectors: 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.inplace_update_support: 0 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.bloom_locality: 0 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.max_successive_merges: 0 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.paranoid_file_checks: 0 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.force_consistency_checks: 1 2023-12-17T12:01:29.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.report_bg_io_stats: 0 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.ttl: 2592000 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.enable_blob_files: false 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.min_blob_size: 0 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.blob_file_size: 268435456 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.blob_file_starting_level: 0 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-17T12:01:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 45710bf0-5e65-4e19-90fe-a989ad0ffe51 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814488748331, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814488750338, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92441, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 305, "table_properties": {"data_size": 90441, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13449, "raw_average_key_size": 50, "raw_value_size": 83105, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 268, "num_filter_entries": 268, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702814488, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "45710bf0-5e65-4e19-90fe-a989ad0ffe51", "db_session_id": "PGD7JN2LI7LV1K97WGXA", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814488750472, "job": 1, "event": "recovery_finished"} 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f883b90000 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: DB pointer 0x55f883b7c000 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 systemd[1]: Started Ceph mon.a for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** DB Stats ** 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-17T12:01:29.182 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: L0 2/0 92.11 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: Sum 2/0 92.11 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:01:29.183 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative compaction: 0.00 GB write, 12.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval compaction: 0.00 GB write, 12.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache BinnedLRUCache@0x55f883af51f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-17T12:01:29.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: starting mon.a rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a@-1(???) e1 preinit fsid 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a@-1(???).mds e1 new map 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a@-1(???).mds e1 print_map 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout: e1 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout: legacy client fscid: -1 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout: No filesystems configured 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-17T12:01:29.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-17T12:01:29.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: monmap e1: 1 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:01:29.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: fsmap 2023-12-17T12:01:29.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: osdmap e1: 0 total, 0 up, 0 in 2023-12-17T12:01:29.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:28 smithi027 ceph-mon[36871]: mgrmap e1: no daemons active 2023-12-17T12:01:29.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.189+0000 7fe19df1d700 1 Processor -- start 2023-12-17T12:01:29.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.189+0000 7fe19df1d700 1 -- start start 2023-12-17T12:01:29.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.189+0000 7fe19df1d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19807b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:29.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.189+0000 7fe19df1d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19807b8a0 con 0x7fe19807ce70 2023-12-17T12:01:29.613 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.190+0000 7fe1977fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19807b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.190+0000 7fe1977fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19807b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40944/0 (socket says 172.21.15.27:40944) 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.190+0000 7fe1977fe700 1 -- 172.21.15.27:0/188249988 learned_addr learned my addr 172.21.15.27:0/188249988 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.190+0000 7fe1977fe700 1 -- 172.21.15.27:0/188249988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19807b9e0 con 0x7fe19807ce70 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.190+0000 7fe1977fe700 1 --2- 172.21.15.27:0/188249988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19807b2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe1880098a0 tx=0x7fe188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ff4e7e00f782a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.191+0000 7fe1967fc700 1 -- 172.21.15.27:0/188249988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe188004160 con 0x7fe19807ce70 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.191+0000 7fe1967fc700 1 -- 172.21.15.27:0/188249988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe1880042c0 con 0x7fe19807ce70 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.191+0000 7fe1967fc700 1 -- 172.21.15.27:0/188249988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe188011940 con 0x7fe19807ce70 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.191+0000 7fe19df1d700 1 -- 172.21.15.27:0/188249988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 msgr2=0x7fe19807b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.191+0000 7fe19df1d700 1 --2- 172.21.15.27:0/188249988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19807b2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe1880098a0 tx=0x7fe188009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.191+0000 7fe19df1d700 1 -- 172.21.15.27:0/188249988 shutdown_connections 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.191+0000 7fe19df1d700 1 --2- 172.21.15.27:0/188249988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19807b2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.191+0000 7fe19df1d700 1 -- 172.21.15.27:0/188249988 >> 172.21.15.27:0/188249988 conn(0x7fe198102170 msgr2=0x7fe198104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.192+0000 7fe19df1d700 1 -- 172.21.15.27:0/188249988 shutdown_connections 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.192+0000 7fe19df1d700 1 -- 172.21.15.27:0/188249988 wait complete. 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.193+0000 7fe19df1d700 1 Processor -- start 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.193+0000 7fe19df1d700 1 -- start start 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.193+0000 7fe19df1d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.193+0000 7fe19df1d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19819cee0 con 0x7fe19807ce70 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.193+0000 7fe1977fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.193+0000 7fe1977fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40952/0 (socket says 172.21.15.27:40952) 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.193+0000 7fe1977fe700 1 -- 172.21.15.27:0/697374902 learned_addr learned my addr 172.21.15.27:0/697374902 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.194+0000 7fe1977fe700 1 -- 172.21.15.27:0/697374902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188009550 con 0x7fe19807ce70 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.194+0000 7fe1977fe700 1 --2- 172.21.15.27:0/697374902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19819c9a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe188003f20 tx=0x7fe188011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.194+0000 7fe194ff9700 1 -- 172.21.15.27:0/697374902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1880036a0 con 0x7fe19807ce70 2023-12-17T12:01:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.194+0000 7fe194ff9700 1 -- 172.21.15.27:0/697374902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe18801b440 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.194+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19819d0e0 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.194+0000 7fe194ff9700 1 -- 172.21.15.27:0/697374902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe188023400 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.194+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19819d580 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.195+0000 7fe194ff9700 1 -- 172.21.15.27:0/697374902 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe188023970 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.195+0000 7fe194ff9700 1 -- 172.21.15.27:0/697374902 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe1880193d0 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.195+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19810cea0 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.199+0000 7fe194ff9700 1 -- 172.21.15.27:0/697374902 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe188023d60 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.273+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fe19819fe10 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.278+0000 7fe194ff9700 1 -- 172.21.15.27:0/697374902 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe188006d60 con 0x7fe19807ce70 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.279+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 msgr2=0x7fe19819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.279+0000 7fe19df1d700 1 --2- 172.21.15.27:0/697374902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19819c9a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe188003f20 tx=0x7fe188011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.279+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 shutdown_connections 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.279+0000 7fe19df1d700 1 --2- 172.21.15.27:0/697374902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe19807ce70 0x7fe19819c9a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:29.616 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.279+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 >> 172.21.15.27:0/697374902 conn(0x7fe198102170 msgr2=0x7fe198102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:29.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.279+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 shutdown_connections 2023-12-17T12:01:29.617 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:29.279+0000 7fe19df1d700 1 -- 172.21.15.27:0/697374902 wait complete. 2023-12-17T12:01:29.617 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-17T12:01:29.647 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:01:29.647 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mgr... 2023-12-17T12:01:29.648 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:9283 ... 2023-12-17T12:01:29.648 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:8765 ... 2023-12-17T12:01:30.025 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.x 2023-12-17T12:01:30.026 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.x.service: Unit ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.x.service not loaded. 2023-12-17T12:01:30.240 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-62030760-9cd3-11ee-95a5-87774f69a715.target.wants/ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-62030760-9cd3-11ee-95a5-87774f69a715@.service. 2023-12-17T12:01:30.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:30 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/697374902' entity='client.admin' 2023-12-17T12:01:30.414 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:30 smithi027 systemd[1]: Starting Ceph mgr.x for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:01:30.813 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-17T12:01:30.813 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2023-12-17T12:01:30.830 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-17T12:01:30.830 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2023-12-17T12:01:30.830 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2023-12-17T12:01:30.830 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2023-12-17T12:01:30.849 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-17T12:01:30.849 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2023-12-17T12:01:30.866 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-17T12:01:30.866 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2023-12-17T12:01:30.866 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2023-12-17T12:01:30.866 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-17T12:01:30.866 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr to start... 2023-12-17T12:01:30.866 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr... 2023-12-17T12:01:31.058 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:30 smithi027 bash[37107]: 64628503b6415e00177383589f7226156abf824ad6df8ab3a7d6ef65b470b672 2023-12-17T12:01:31.058 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:30 smithi027 systemd[1]: Started Ceph mgr.x for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:01:31.617 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:31 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3771694958' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-17T12:01:31.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:31.364+0000 7fad1d292200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-17T12:01:31.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:31.493+0000 7fad1d292200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-17T12:01:31.686 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:31.686 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:01:31.686 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "62030760-9cd3-11ee-95a5-87774f69a715", 2023-12-17T12:01:31.686 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2023-12-17T12:01:31.686 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-17T12:01:31.686 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-12-17T12:01:31.687 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-17T12:01:31.688 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:31.688 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-17T12:01:31.688 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-17T12:01:31.688 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:31.688 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-17T12:01:31.688 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:31.688 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-17T12:01:31.696 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-17T12:01:31.696 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-17T12:01:31.696 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-17T12:01:31.696 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-17T12:01:31.696 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-17T12:01:31.696 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-17T12:01:31.697 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2023-12-17T12:01:24.597006+0000", 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:31.698 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.326+0000 7f933fa29700 1 Processor -- start 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.326+0000 7f933fa29700 1 -- start start 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.327+0000 7f933fa29700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f9338106f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.327+0000 7f933fa29700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338107530 con 0x7f9338106b40 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.327+0000 7f933d7c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f9338106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.327+0000 7f933d7c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f9338106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40988/0 (socket says 172.21.15.27:40988) 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.327+0000 7f933d7c5700 1 -- 172.21.15.27:0/3136520606 learned_addr learned my addr 172.21.15.27:0/3136520606 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.328+0000 7f933d7c5700 1 -- 172.21.15.27:0/3136520606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338107d40 con 0x7f9338106b40 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.328+0000 7f933d7c5700 1 --2- 172.21.15.27:0/3136520606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f9338106f60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93340098a0 tx=0x7f9334009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3957f4ae95df8ac server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.328+0000 7f932ffff700 1 -- 172.21.15.27:0/3136520606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9334004160 con 0x7f9338106b40 2023-12-17T12:01:31.699 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.328+0000 7f932ffff700 1 -- 172.21.15.27:0/3136520606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f93340042c0 con 0x7f9338106b40 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.328+0000 7f932ffff700 1 -- 172.21.15.27:0/3136520606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9334011940 con 0x7f9338106b40 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.329+0000 7f933fa29700 1 -- 172.21.15.27:0/3136520606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 msgr2=0x7f9338106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.329+0000 7f933fa29700 1 --2- 172.21.15.27:0/3136520606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f9338106f60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93340098a0 tx=0x7f9334009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.329+0000 7f933fa29700 1 -- 172.21.15.27:0/3136520606 shutdown_connections 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.329+0000 7f933fa29700 1 --2- 172.21.15.27:0/3136520606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f9338106f60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.329+0000 7f933fa29700 1 -- 172.21.15.27:0/3136520606 >> 172.21.15.27:0/3136520606 conn(0x7f9338102140 msgr2=0x7f9338104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.329+0000 7f933fa29700 1 -- 172.21.15.27:0/3136520606 shutdown_connections 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.329+0000 7f933fa29700 1 -- 172.21.15.27:0/3136520606 wait complete. 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.330+0000 7f933fa29700 1 Processor -- start 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.330+0000 7f933fa29700 1 -- start start 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.330+0000 7f933fa29700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f933807cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.331+0000 7f933fa29700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933807d010 con 0x7f9338106b40 2023-12-17T12:01:31.700 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.331+0000 7f933d7c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f933807cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.331+0000 7f933d7c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f933807cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40990/0 (socket says 172.21.15.27:40990) 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.331+0000 7f933d7c5700 1 -- 172.21.15.27:0/3771694958 learned_addr learned my addr 172.21.15.27:0/3771694958 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.331+0000 7f933d7c5700 1 -- 172.21.15.27:0/3771694958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334009550 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.332+0000 7f933d7c5700 1 --2- 172.21.15.27:0/3771694958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f933807cad0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9334003f20 tx=0x7f9334011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.332+0000 7f932e7fc700 1 -- 172.21.15.27:0/3771694958 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93340036a0 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.332+0000 7f932e7fc700 1 -- 172.21.15.27:0/3771694958 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f933401b440 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.332+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933807b250 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.332+0000 7f932e7fc700 1 -- 172.21.15.27:0/3771694958 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9334023400 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.332+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933807b6f0 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.332+0000 7f932e7fc700 1 -- 172.21.15.27:0/3771694958 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9334023970 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.333+0000 7f932e7fc700 1 -- 172.21.15.27:0/3771694958 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f933402cab0 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.333+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9338062c00 con 0x7f9338106b40 2023-12-17T12:01:31.701 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.336+0000 7f932e7fc700 1 -- 172.21.15.27:0/3771694958 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f933401b5b0 con 0x7f9338106b40 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.414+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f933807b990 con 0x7f9338106b40 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.414+0000 7f932e7fc700 1 -- 172.21.15.27:0/3771694958 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9334023d10 con 0x7f9338106b40 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.415+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 msgr2=0x7f933807cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.415+0000 7f933fa29700 1 --2- 172.21.15.27:0/3771694958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f933807cad0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9334003f20 tx=0x7f9334011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.415+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 shutdown_connections 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.415+0000 7f933fa29700 1 --2- 172.21.15.27:0/3771694958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9338106b40 0x7f933807cad0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.415+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 >> 172.21.15.27:0/3771694958 conn(0x7f9338102140 msgr2=0x7f9338102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.415+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 shutdown_connections 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:31.416+0000 7f933fa29700 1 -- 172.21.15.27:0/3771694958 wait complete. 2023-12-17T12:01:31.702 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (1/15)... 2023-12-17T12:01:31.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:31.615+0000 7fad1d292200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-17T12:01:31.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:31.733+0000 7fad1d292200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-17T12:01:32.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:32 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:32.628+0000 7fad1d292200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:01:33.204 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:32 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:32.941+0000 7fad1d292200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-17T12:01:33.204 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:33.056+0000 7fad1d292200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-17T12:01:33.585 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:33.203+0000 7fad1d292200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-17T12:01:33.585 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:33.406+0000 7fad1d292200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-17T12:01:33.879 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:33.583+0000 7fad1d292200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-17T12:01:33.879 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:33.702+0000 7fad1d292200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-17T12:01:34.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:34.592 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:01:34.593 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:34 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:34.223+0000 7fad1d292200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "62030760-9cd3-11ee-95a5-87774f69a715", 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2023-12-17T12:01:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:34.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-17T12:01:34.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-17T12:01:34.599 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-17T12:01:34.599 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-17T12:01:34.599 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-17T12:01:34.606 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2023-12-17T12:01:24.597006+0000", 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.159+0000 7fc4aeb14700 1 Processor -- start 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.159+0000 7fc4aeb14700 1 -- start start 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.159+0000 7fc4aeb14700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a8106f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:34.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.159+0000 7fc4aeb14700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8107530 con 0x7fc4a8106b40 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.160+0000 7fc4ac8b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a8106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.160+0000 7fc4ac8b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a8106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40996/0 (socket says 172.21.15.27:40996) 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.160+0000 7fc4ac8b0700 1 -- 172.21.15.27:0/4113082009 learned_addr learned my addr 172.21.15.27:0/4113082009 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.160+0000 7fc4ac8b0700 1 -- 172.21.15.27:0/4113082009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a8107d40 con 0x7fc4a8106b40 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.161+0000 7fc4ac8b0700 1 --2- 172.21.15.27:0/4113082009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a8106f60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0098a0 tx=0x7fc49c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3fc0abd607294d server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.161+0000 7fc4a77fe700 1 -- 172.21.15.27:0/4113082009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc49c004160 con 0x7fc4a8106b40 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.161+0000 7fc4a77fe700 1 -- 172.21.15.27:0/4113082009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc49c0042c0 con 0x7fc4a8106b40 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.161+0000 7fc4a77fe700 1 -- 172.21.15.27:0/4113082009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc49c011940 con 0x7fc4a8106b40 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.162+0000 7fc4aeb14700 1 -- 172.21.15.27:0/4113082009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 msgr2=0x7fc4a8106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.162+0000 7fc4aeb14700 1 --2- 172.21.15.27:0/4113082009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a8106f60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0098a0 tx=0x7fc49c009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.162+0000 7fc4aeb14700 1 -- 172.21.15.27:0/4113082009 shutdown_connections 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.162+0000 7fc4aeb14700 1 --2- 172.21.15.27:0/4113082009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a8106f60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:34.608 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.162+0000 7fc4aeb14700 1 -- 172.21.15.27:0/4113082009 >> 172.21.15.27:0/4113082009 conn(0x7fc4a8102140 msgr2=0x7fc4a8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.162+0000 7fc4aeb14700 1 -- 172.21.15.27:0/4113082009 shutdown_connections 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.162+0000 7fc4aeb14700 1 -- 172.21.15.27:0/4113082009 wait complete. 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.163+0000 7fc4aeb14700 1 Processor -- start 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.163+0000 7fc4aeb14700 1 -- start start 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.164+0000 7fc4aeb14700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.164+0000 7fc4aeb14700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a819ad00 con 0x7fc4a8106b40 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.164+0000 7fc4ac8b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.164+0000 7fc4ac8b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40998/0 (socket says 172.21.15.27:40998) 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.164+0000 7fc4ac8b0700 1 -- 172.21.15.27:0/721129960 learned_addr learned my addr 172.21.15.27:0/721129960 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.164+0000 7fc4ac8b0700 1 -- 172.21.15.27:0/721129960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c009550 con 0x7fc4a8106b40 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.164+0000 7fc4ac8b0700 1 --2- 172.21.15.27:0/721129960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a819a7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc49c003f20 tx=0x7fc49c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.165+0000 7fc4a5ffb700 1 -- 172.21.15.27:0/721129960 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc49c0036a0 con 0x7fc4a8106b40 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.165+0000 7fc4a5ffb700 1 -- 172.21.15.27:0/721129960 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc49c01b440 con 0x7fc4a8106b40 2023-12-17T12:01:34.609 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.165+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a819af00 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.165+0000 7fc4a5ffb700 1 -- 172.21.15.27:0/721129960 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc49c023400 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.165+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a819b3a0 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.165+0000 7fc4a5ffb700 1 -- 172.21.15.27:0/721129960 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc49c0239c0 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.165+0000 7fc4a5ffb700 1 -- 172.21.15.27:0/721129960 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc49c02ce40 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.166+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a8062c00 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.169+0000 7fc4a5ffb700 1 -- 172.21.15.27:0/721129960 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc49c023d60 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.248+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc4a810af70 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.248+0000 7fc4a5ffb700 1 -- 172.21.15.27:0/721129960 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc49c01b5b0 con 0x7fc4a8106b40 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.250+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 msgr2=0x7fc4a819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.250+0000 7fc4aeb14700 1 --2- 172.21.15.27:0/721129960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a819a7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc49c003f20 tx=0x7fc49c011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.250+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 shutdown_connections 2023-12-17T12:01:34.610 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.250+0000 7fc4aeb14700 1 --2- 172.21.15.27:0/721129960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4a8106b40 0x7fc4a819a7c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:34.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.250+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 >> 172.21.15.27:0/721129960 conn(0x7fc4a8102140 msgr2=0x7fc4a8102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:34.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.250+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 shutdown_connections 2023-12-17T12:01:34.611 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:34.250+0000 7fc4aeb14700 1 -- 172.21.15.27:0/721129960 wait complete. 2023-12-17T12:01:34.611 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (2/15)... 2023-12-17T12:01:34.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:34 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/721129960' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-17T12:01:34.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:34 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:34.608+0000 7fad1d292200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-17T12:01:35.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:34 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:34.978+0000 7fad1d292200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-17T12:01:35.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:35 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:35.107+0000 7fad1d292200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-17T12:01:36.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:36 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:36.103+0000 7fad1d292200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-17T12:01:36.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:36 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:36.233+0000 7fad1d292200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-17T12:01:36.754 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:36 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:36.436+0000 7fad1d292200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-17T12:01:37.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:37 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1296305298' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-17T12:01:37.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:37 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:37.334+0000 7fad1d292200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "62030760-9cd3-11ee-95a5-87774f69a715", 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-17T12:01:37.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-17T12:01:37.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-17T12:01:37.452 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-17T12:01:37.453 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2023-12-17T12:01:24.597006+0000", 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.008+0000 7f42b0905700 1 Processor -- start 2023-12-17T12:01:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.008+0000 7f42b0905700 1 -- start start 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.008+0000 7f42b0905700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac106f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.008+0000 7f42b0905700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac107530 con 0x7f42ac106b40 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.009+0000 7f42aa59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.009+0000 7f42aa59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49900/0 (socket says 172.21.15.27:49900) 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.009+0000 7f42aa59c700 1 -- 172.21.15.27:0/1538319745 learned_addr learned my addr 172.21.15.27:0/1538319745 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.009+0000 7f42aa59c700 1 -- 172.21.15.27:0/1538319745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac107d40 con 0x7f42ac106b40 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.009+0000 7f42aa59c700 1 --2- 172.21.15.27:0/1538319745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac106f60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f429c0098a0 tx=0x7f429c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ea9a980eef3dad server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42a959a700 1 -- 172.21.15.27:0/1538319745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f429c004160 con 0x7f42ac106b40 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42a959a700 1 -- 172.21.15.27:0/1538319745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f429c0042c0 con 0x7f42ac106b40 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42a959a700 1 -- 172.21.15.27:0/1538319745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f429c011940 con 0x7f42ac106b40 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42b0905700 1 -- 172.21.15.27:0/1538319745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 msgr2=0x7f42ac106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42b0905700 1 --2- 172.21.15.27:0/1538319745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac106f60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f429c0098a0 tx=0x7f429c009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42b0905700 1 -- 172.21.15.27:0/1538319745 shutdown_connections 2023-12-17T12:01:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42b0905700 1 --2- 172.21.15.27:0/1538319745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac106f60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42b0905700 1 -- 172.21.15.27:0/1538319745 >> 172.21.15.27:0/1538319745 conn(0x7f42ac102140 msgr2=0x7f42ac104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.010+0000 7f42b0905700 1 -- 172.21.15.27:0/1538319745 shutdown_connections 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.011+0000 7f42b0905700 1 -- 172.21.15.27:0/1538319745 wait complete. 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.012+0000 7f42b0905700 1 Processor -- start 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.012+0000 7f42b0905700 1 -- start start 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.013+0000 7f42b0905700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.013+0000 7f42b0905700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac19ad00 con 0x7f42ac106b40 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.013+0000 7f42aa59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.013+0000 7f42aa59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49912/0 (socket says 172.21.15.27:49912) 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.013+0000 7f42aa59c700 1 -- 172.21.15.27:0/1296305298 learned_addr learned my addr 172.21.15.27:0/1296305298 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.013+0000 7f42aa59c700 1 -- 172.21.15.27:0/1296305298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c009550 con 0x7f42ac106b40 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.014+0000 7f42aa59c700 1 --2- 172.21.15.27:0/1296305298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac19a7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f429c003f20 tx=0x7f429c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.014+0000 7f429b7fe700 1 -- 172.21.15.27:0/1296305298 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f429c0036a0 con 0x7f42ac106b40 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.014+0000 7f429b7fe700 1 -- 172.21.15.27:0/1296305298 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f429c01b440 con 0x7f42ac106b40 2023-12-17T12:01:37.456 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.014+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac19af00 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.014+0000 7f429b7fe700 1 -- 172.21.15.27:0/1296305298 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f429c023400 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.014+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac19b3a0 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.015+0000 7f429b7fe700 1 -- 172.21.15.27:0/1296305298 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f429c023970 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.015+0000 7f429b7fe700 1 -- 172.21.15.27:0/1296305298 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f429c02cdf0 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.015+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac062c00 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.018+0000 7f429b7fe700 1 -- 172.21.15.27:0/1296305298 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f429c023d60 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.096+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f42ac10af70 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.097+0000 7f429b7fe700 1 -- 172.21.15.27:0/1296305298 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f429c01b5b0 con 0x7f42ac106b40 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.098+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 msgr2=0x7f42ac19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.098+0000 7f42b0905700 1 --2- 172.21.15.27:0/1296305298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac19a7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f429c003f20 tx=0x7f429c011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.098+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 shutdown_connections 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.098+0000 7f42b0905700 1 --2- 172.21.15.27:0/1296305298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42ac106b40 0x7f42ac19a7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.098+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 >> 172.21.15.27:0/1296305298 conn(0x7f42ac102140 msgr2=0x7f42ac102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:37.457 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.098+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 shutdown_connections 2023-12-17T12:01:37.458 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:37.098+0000 7f42b0905700 1 -- 172.21.15.27:0/1296305298 wait complete. 2023-12-17T12:01:37.458 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (3/15)... 2023-12-17T12:01:37.853 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:37 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:37.583+0000 7fad1d292200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-17T12:01:38.162 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:37 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:37.851+0000 7fad1d292200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-17T12:01:39.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:39.042+0000 7fad1d292200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-17T12:01:39.825 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:39.412+0000 7fad1d292200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:01:39.825 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:39.543+0000 7fad1d292200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-17T12:01:40.157 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:39.823+0000 7fad1d292200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-17T12:01:40.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: Activating manager daemon x 2023-12-17T12:01:40.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: mgrmap e2: x(active, starting, since 0.00436941s) 2023-12-17T12:01:40.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-17T12:01:40.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-17T12:01:40.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-17T12:01:40.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:01:40.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-17T12:01:40.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: Manager daemon x is now available 2023-12-17T12:01:40.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-17T12:01:40.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-17T12:01:40.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:39 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' 2023-12-17T12:01:40.329 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "62030760-9cd3-11ee-95a5-87774f69a715", 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:40.331 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-17T12:01:40.339 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-17T12:01:40.339 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-17T12:01:40.339 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-17T12:01:40.339 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-17T12:01:40.339 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-17T12:01:40.340 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2023-12-17T12:01:24.597006+0000", 2023-12-17T12:01:40.341 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.868+0000 7fbaa241e700 1 Processor -- start 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.868+0000 7fbaa241e700 1 -- start start 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.868+0000 7fbaa241e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.868+0000 7fbaa241e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c106450 con 0x7fba9c105a60 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.869+0000 7fba9bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.869+0000 7fba9bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49964/0 (socket says 172.21.15.27:49964) 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.869+0000 7fba9bfff700 1 -- 172.21.15.27:0/1507962029 learned_addr learned my addr 172.21.15.27:0/1507962029 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.869+0000 7fba9bfff700 1 -- 172.21.15.27:0/1507962029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c106c60 con 0x7fba9c105a60 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.870+0000 7fba9bfff700 1 --2- 172.21.15.27:0/1507962029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c105e80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0098a0 tx=0x7fba8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a83c2d7b6cb47b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:40.342 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.870+0000 7fba9affd700 1 -- 172.21.15.27:0/1507962029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba8c004160 con 0x7fba9c105a60 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.870+0000 7fba9affd700 1 -- 172.21.15.27:0/1507962029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fba8c0042c0 con 0x7fba9c105a60 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.870+0000 7fba9affd700 1 -- 172.21.15.27:0/1507962029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba8c011940 con 0x7fba9c105a60 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.870+0000 7fbaa241e700 1 -- 172.21.15.27:0/1507962029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 msgr2=0x7fba9c105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.870+0000 7fbaa241e700 1 --2- 172.21.15.27:0/1507962029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c105e80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0098a0 tx=0x7fba8c009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.871+0000 7fbaa241e700 1 -- 172.21.15.27:0/1507962029 shutdown_connections 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.871+0000 7fbaa241e700 1 --2- 172.21.15.27:0/1507962029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c105e80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.871+0000 7fbaa241e700 1 -- 172.21.15.27:0/1507962029 >> 172.21.15.27:0/1507962029 conn(0x7fba9c101020 msgr2=0x7fba9c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.871+0000 7fbaa241e700 1 -- 172.21.15.27:0/1507962029 shutdown_connections 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.871+0000 7fbaa241e700 1 -- 172.21.15.27:0/1507962029 wait complete. 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.873+0000 7fbaa241e700 1 Processor -- start 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.873+0000 7fbaa241e700 1 -- start start 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.873+0000 7fbaa241e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.873+0000 7fbaa241e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c19cf90 con 0x7fba9c105a60 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.874+0000 7fba9bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.874+0000 7fba9bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49988/0 (socket says 172.21.15.27:49988) 2023-12-17T12:01:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.874+0000 7fba9bfff700 1 -- 172.21.15.27:0/1252860742 learned_addr learned my addr 172.21.15.27:0/1252860742 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.875+0000 7fba9bfff700 1 -- 172.21.15.27:0/1252860742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c009550 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.875+0000 7fba9bfff700 1 --2- 172.21.15.27:0/1252860742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c19ca50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fba8c003f20 tx=0x7fba8c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.876+0000 7fba997fa700 1 -- 172.21.15.27:0/1252860742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba8c0036a0 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.876+0000 7fba997fa700 1 -- 172.21.15.27:0/1252860742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fba8c003c70 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.876+0000 7fba997fa700 1 -- 172.21.15.27:0/1252860742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba8c01be80 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.876+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c19d190 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.876+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c19d630 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.877+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9c079f80 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.878+0000 7fba997fa700 1 -- 172.21.15.27:0/1252860742 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 2) v1 ==== 45336+0+0 (secure 0 0 0) 0x7fba8c01b440 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.878+0000 7fba997fa700 1 -- 172.21.15.27:0/1252860742 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba8c04c400 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.881+0000 7fba997fa700 1 -- 172.21.15.27:0/1252860742 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fba8c00b9d0 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.960+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fba9c19fec0 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.961+0000 7fba997fa700 1 -- 172.21.15.27:0/1252860742 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fba8c00b3e0 con 0x7fba9c105a60 2023-12-17T12:01:40.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.961+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 msgr2=0x7fba9c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:40.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.961+0000 7fbaa241e700 1 --2- 172.21.15.27:0/1252860742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c19ca50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fba8c003f20 tx=0x7fba8c011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:40.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.961+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 shutdown_connections 2023-12-17T12:01:40.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.961+0000 7fbaa241e700 1 --2- 172.21.15.27:0/1252860742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c105a60 0x7fba9c19ca50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:40.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.961+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 >> 172.21.15.27:0/1252860742 conn(0x7fba9c101020 msgr2=0x7fba9c10a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:40.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.962+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 shutdown_connections 2023-12-17T12:01:40.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:39.962+0000 7fbaa241e700 1 -- 172.21.15.27:0/1252860742 wait complete. 2023-12-17T12:01:40.345 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (4/15)... 2023-12-17T12:01:41.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:40 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' 2023-12-17T12:01:41.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:40 smithi027 ceph-mon[36871]: from='mgr.14100 172.21.15.27:0/3124646467' entity='mgr.x' 2023-12-17T12:01:41.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:40 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1252860742' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-17T12:01:41.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:40 smithi027 ceph-mon[36871]: mgrmap e3: x(active, since 1.00887s) 2023-12-17T12:01:43.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:42 smithi027 ceph-mon[36871]: mgrmap e4: x(active, since 2s) 2023-12-17T12:01:43.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:43.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:01:43.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "62030760-9cd3-11ee-95a5-87774f69a715", 2023-12-17T12:01:43.367 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-17T12:01:43.368 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-17T12:01:43.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:43.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-17T12:01:43.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:43.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-17T12:01:43.369 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-17T12:01:43.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-17T12:01:43.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-17T12:01:43.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-17T12:01:43.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-17T12:01:43.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:43.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-17T12:01:43.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:43.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2023-12-17T12:01:24.597006+0000", 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2023-12-17T12:01:43.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.746+0000 7fa70a2c5700 1 Processor -- start 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.746+0000 7fa70a2c5700 1 -- start start 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.747+0000 7fa70a2c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa70407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.747+0000 7fa70a2c5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70407b8c0 con 0x7fa70407ce90 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.747+0000 7fa703fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa70407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.747+0000 7fa703fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa70407b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50012/0 (socket says 172.21.15.27:50012) 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.747+0000 7fa703fff700 1 -- 172.21.15.27:0/3275495335 learned_addr learned my addr 172.21.15.27:0/3275495335 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.747+0000 7fa703fff700 1 -- 172.21.15.27:0/3275495335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70407ba00 con 0x7fa70407ce90 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.747+0000 7fa703fff700 1 --2- 172.21.15.27:0/3275495335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa70407b2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40098a0 tx=0x7fa6f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75519f7d2e37135d server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.748+0000 7fa702ffd700 1 -- 172.21.15.27:0/3275495335 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6f4004160 con 0x7fa70407ce90 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.748+0000 7fa702ffd700 1 -- 172.21.15.27:0/3275495335 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa6f40042c0 con 0x7fa70407ce90 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.748+0000 7fa702ffd700 1 -- 172.21.15.27:0/3275495335 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6f4011940 con 0x7fa70407ce90 2023-12-17T12:01:43.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.748+0000 7fa70a2c5700 1 -- 172.21.15.27:0/3275495335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 msgr2=0x7fa70407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.748+0000 7fa70a2c5700 1 --2- 172.21.15.27:0/3275495335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa70407b2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40098a0 tx=0x7fa6f4009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.748+0000 7fa70a2c5700 1 -- 172.21.15.27:0/3275495335 shutdown_connections 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.748+0000 7fa70a2c5700 1 --2- 172.21.15.27:0/3275495335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa70407b2f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.748+0000 7fa70a2c5700 1 -- 172.21.15.27:0/3275495335 >> 172.21.15.27:0/3275495335 conn(0x7fa704102140 msgr2=0x7fa704104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.749+0000 7fa70a2c5700 1 -- 172.21.15.27:0/3275495335 shutdown_connections 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.749+0000 7fa70a2c5700 1 -- 172.21.15.27:0/3275495335 wait complete. 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.749+0000 7fa70a2c5700 1 Processor -- start 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.750+0000 7fa70a2c5700 1 -- start start 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.750+0000 7fa70a2c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa7041a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.750+0000 7fa70a2c5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7041a12f0 con 0x7fa70407ce90 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.750+0000 7fa703fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa7041a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.750+0000 7fa703fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa7041a0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50020/0 (socket says 172.21.15.27:50020) 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.750+0000 7fa703fff700 1 -- 172.21.15.27:0/1469281108 learned_addr learned my addr 172.21.15.27:0/1469281108 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:43.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.751+0000 7fa703fff700 1 -- 172.21.15.27:0/1469281108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4009550 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.751+0000 7fa703fff700 1 --2- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa7041a0db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4003f20 tx=0x7fa6f4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.751+0000 7fa7017fa700 1 -- 172.21.15.27:0/1469281108 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6f40036a0 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.751+0000 7fa7017fa700 1 -- 172.21.15.27:0/1469281108 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa6f401b440 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.751+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7041a14f0 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.751+0000 7fa7017fa700 1 -- 172.21.15.27:0/1469281108 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6f4023400 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.751+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7041a1990 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.752+0000 7fa7017fa700 1 -- 172.21.15.27:0/1469281108 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fa6f4003800 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.752+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70419a5e0 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.753+0000 7fa7017fa700 1 --2- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fa6ec038820 0x7fa6ec03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.753+0000 7fa7017fa700 1 -- 172.21.15.27:0/1469281108 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa6f404c780 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.753+0000 7fa7037fe700 1 --2- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fa6ec038820 0x7fa6ec03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.754+0000 7fa7037fe700 1 --2- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fa6ec038820 0x7fa6ec03ace0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8006fd0 tx=0x7fa6f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:42.759+0000 7fa7017fa700 1 -- 172.21.15.27:0/1469281108 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa6f400ba30 con 0x7fa70407ce90 2023-12-17T12:01:43.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.051+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa70402d510 con 0x7fa70407ce90 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.052+0000 7fa7017fa700 1 -- 172.21.15.27:0/1469281108 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fa6f4019a90 con 0x7fa70407ce90 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.054+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fa6ec038820 msgr2=0x7fa6ec03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.054+0000 7fa70a2c5700 1 --2- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fa6ec038820 0x7fa6ec03ace0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8006fd0 tx=0x7fa6f8006e40 comp rx=0 tx=0).stop 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.054+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 msgr2=0x7fa7041a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.054+0000 7fa70a2c5700 1 --2- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa7041a0db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4003f20 tx=0x7fa6f4011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.054+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 shutdown_connections 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.054+0000 7fa70a2c5700 1 --2- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fa6ec038820 0x7fa6ec03ace0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.054+0000 7fa70a2c5700 1 --2- 172.21.15.27:0/1469281108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa70407ce90 0x7fa7041a0db0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.054+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 >> 172.21.15.27:0/1469281108 conn(0x7fa704102140 msgr2=0x7fa704104530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.055+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 shutdown_connections 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.055+0000 7fa70a2c5700 1 -- 172.21.15.27:0/1469281108 wait complete. 2023-12-17T12:01:43.383 INFO:teuthology.orchestra.run.smithi027.stdout:mgr is available 2023-12-17T12:01:44.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:43 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1469281108' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-17T12:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.763+0000 7fd5fbf54700 1 Processor -- start 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.763+0000 7fd5fbf54700 1 -- start start 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.763+0000 7fd5fbf54700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.763+0000 7fd5fbf54700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f4107530 con 0x7fd5f4106b40 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.763+0000 7fd5f9cf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.764+0000 7fd5f9cf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f4106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50034/0 (socket says 172.21.15.27:50034) 2023-12-17T12:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.764+0000 7fd5f9cf0700 1 -- 172.21.15.27:0/3520940199 learned_addr learned my addr 172.21.15.27:0/3520940199 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.764+0000 7fd5f9cf0700 1 -- 172.21.15.27:0/3520940199 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4107d40 con 0x7fd5f4106b40 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.764+0000 7fd5f9cf0700 1 --2- 172.21.15.27:0/3520940199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f4106f60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00098a0 tx=0x7fd5f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ab292b658b8f50 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.764+0000 7fd5f8cee700 1 -- 172.21.15.27:0/3520940199 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5f0004160 con 0x7fd5f4106b40 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.764+0000 7fd5f8cee700 1 -- 172.21.15.27:0/3520940199 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd5f00042c0 con 0x7fd5f4106b40 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.764+0000 7fd5f8cee700 1 -- 172.21.15.27:0/3520940199 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5f0011940 con 0x7fd5f4106b40 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.765+0000 7fd5fbf54700 1 -- 172.21.15.27:0/3520940199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 msgr2=0x7fd5f4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.765+0000 7fd5fbf54700 1 --2- 172.21.15.27:0/3520940199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f4106f60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00098a0 tx=0x7fd5f0009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.765+0000 7fd5fbf54700 1 -- 172.21.15.27:0/3520940199 shutdown_connections 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.765+0000 7fd5fbf54700 1 --2- 172.21.15.27:0/3520940199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f4106f60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.765+0000 7fd5fbf54700 1 -- 172.21.15.27:0/3520940199 >> 172.21.15.27:0/3520940199 conn(0x7fd5f4102140 msgr2=0x7fd5f4104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.765+0000 7fd5fbf54700 1 -- 172.21.15.27:0/3520940199 shutdown_connections 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.765+0000 7fd5fbf54700 1 -- 172.21.15.27:0/3520940199 wait complete. 2023-12-17T12:01:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.766+0000 7fd5fbf54700 1 Processor -- start 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.767+0000 7fd5fbf54700 1 -- start start 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.767+0000 7fd5fbf54700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.767+0000 7fd5fbf54700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f419ad00 con 0x7fd5f4106b40 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.767+0000 7fd5f9cf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.767+0000 7fd5f9cf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50038/0 (socket says 172.21.15.27:50038) 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.767+0000 7fd5f9cf0700 1 -- 172.21.15.27:0/393377911 learned_addr learned my addr 172.21.15.27:0/393377911 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.767+0000 7fd5f9cf0700 1 -- 172.21.15.27:0/393377911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f0009550 con 0x7fd5f4106b40 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.768+0000 7fd5f9cf0700 1 --2- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f419a7c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0003f20 tx=0x7fd5f0011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.768+0000 7fd5eaffd700 1 -- 172.21.15.27:0/393377911 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5f00036a0 con 0x7fd5f4106b40 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.768+0000 7fd5eaffd700 1 -- 172.21.15.27:0/393377911 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd5f001b440 con 0x7fd5f4106b40 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.768+0000 7fd5eaffd700 1 -- 172.21.15.27:0/393377911 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5f0023400 con 0x7fd5f4106b40 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.768+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f419af00 con 0x7fd5f4106b40 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.768+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f419b3a0 con 0x7fd5f4106b40 2023-12-17T12:01:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.769+0000 7fd5eaffd700 1 -- 172.21.15.27:0/393377911 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fd5f0003800 con 0x7fd5f4106b40 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.769+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f410af70 con 0x7fd5f4106b40 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.770+0000 7fd5eaffd700 1 --2- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fd5e00387d0 0x7fd5e003ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.770+0000 7fd5eaffd700 1 -- 172.21.15.27:0/393377911 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd5f004c790 con 0x7fd5f4106b40 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.770+0000 7fd5f94ef700 1 --2- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fd5e00387d0 0x7fd5e003ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.771+0000 7fd5f94ef700 1 --2- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fd5e00387d0 0x7fd5e003ac90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.775+0000 7fd5eaffd700 1 -- 172.21.15.27:0/393377911 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd5f000b6b0 con 0x7fd5f4106b40 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.985+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd5f419dc30 con 0x7fd5f4106b40 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.986+0000 7fd5eaffd700 1 -- 172.21.15.27:0/393377911 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd5f001ae60 con 0x7fd5f4106b40 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fd5e00387d0 msgr2=0x7fd5e003ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 --2- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fd5e00387d0 0x7fd5e003ac90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4006e40 comp rx=0 tx=0).stop 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 msgr2=0x7fd5f419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 --2- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f419a7c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0003f20 tx=0x7fd5f0011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:44.320 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 shutdown_connections 2023-12-17T12:01:44.321 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 --2- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7fd5e00387d0 0x7fd5e003ac90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:44.321 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 --2- 172.21.15.27:0/393377911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5f4106b40 0x7fd5f419a7c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:44.321 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 >> 172.21.15.27:0/393377911 conn(0x7fd5f4102140 msgr2=0x7fd5f4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:44.321 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 shutdown_connections 2023-12-17T12:01:44.321 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:43.989+0000 7fd5fbf54700 1 -- 172.21.15.27:0/393377911 wait complete. 2023-12-17T12:01:44.321 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling cephadm module... 2023-12-17T12:01:45.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:44 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/393377911' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-17T12:01:46.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:45 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2227222806' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-17T12:01:46.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:45 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: ignoring --setuser ceph since I am not root 2023-12-17T12:01:46.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:45 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: ignoring --setgroup ceph since I am not root 2023-12-17T12:01:46.237 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.723+0000 7f3b8546f700 1 Processor -- start 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.724+0000 7f3b8546f700 1 -- start start 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.724+0000 7f3b8546f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b80106f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.724+0000 7f3b8546f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80107540 con 0x7f3b80106b50 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.724+0000 7f3b7effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b80106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.724+0000 7f3b7effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b80106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50046/0 (socket says 172.21.15.27:50046) 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.724+0000 7f3b7effd700 1 -- 172.21.15.27:0/2514596456 learned_addr learned my addr 172.21.15.27:0/2514596456 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.725+0000 7f3b7effd700 1 -- 172.21.15.27:0/2514596456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b80107d50 con 0x7f3b80106b50 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.725+0000 7f3b7effd700 1 --2- 172.21.15.27:0/2514596456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b80106f70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3b700098a0 tx=0x7f3b70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19afeb787294118 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.725+0000 7f3b7dffb700 1 -- 172.21.15.27:0/2514596456 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b70004160 con 0x7f3b80106b50 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.725+0000 7f3b7dffb700 1 -- 172.21.15.27:0/2514596456 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3b700042c0 con 0x7f3b80106b50 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.725+0000 7f3b7dffb700 1 -- 172.21.15.27:0/2514596456 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b70011940 con 0x7f3b80106b50 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.726+0000 7f3b8546f700 1 -- 172.21.15.27:0/2514596456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 msgr2=0x7f3b80106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:46.238 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.726+0000 7f3b8546f700 1 --2- 172.21.15.27:0/2514596456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b80106f70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3b700098a0 tx=0x7f3b70009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.726+0000 7f3b8546f700 1 -- 172.21.15.27:0/2514596456 shutdown_connections 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.726+0000 7f3b8546f700 1 --2- 172.21.15.27:0/2514596456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b80106f70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.726+0000 7f3b8546f700 1 -- 172.21.15.27:0/2514596456 >> 172.21.15.27:0/2514596456 conn(0x7f3b80102170 msgr2=0x7f3b80104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.726+0000 7f3b8546f700 1 -- 172.21.15.27:0/2514596456 shutdown_connections 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.726+0000 7f3b8546f700 1 -- 172.21.15.27:0/2514596456 wait complete. 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.727+0000 7f3b8546f700 1 Processor -- start 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.727+0000 7f3b8546f700 1 -- start start 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.727+0000 7f3b8546f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b8019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.727+0000 7f3b8546f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8019cef0 con 0x7f3b80106b50 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.728+0000 7f3b7effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b8019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.728+0000 7f3b7effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b8019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50050/0 (socket says 172.21.15.27:50050) 2023-12-17T12:01:46.239 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.728+0000 7f3b7effd700 1 -- 172.21.15.27:0/2227222806 learned_addr learned my addr 172.21.15.27:0/2227222806 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.728+0000 7f3b7effd700 1 -- 172.21.15.27:0/2227222806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70009550 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.728+0000 7f3b7effd700 1 --2- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b8019c9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b70003f20 tx=0x7f3b70011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.729+0000 7f3b67fff700 1 -- 172.21.15.27:0/2227222806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b700036a0 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.729+0000 7f3b67fff700 1 -- 172.21.15.27:0/2227222806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3b7001b440 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.729+0000 7f3b8546f700 1 -- 172.21.15.27:0/2227222806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8019d0f0 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.729+0000 7f3b67fff700 1 -- 172.21.15.27:0/2227222806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b70023400 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.729+0000 7f3b8546f700 1 -- 172.21.15.27:0/2227222806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8019d590 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.730+0000 7f3b67fff700 1 -- 172.21.15.27:0/2227222806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f3b7001b5b0 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.730+0000 7f3b65ffb700 1 -- 172.21.15.27:0/2227222806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b600052f0 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.730+0000 7f3b67fff700 1 --2- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f3b680387d0 0x7f3b6803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.730+0000 7f3b67fff700 1 -- 172.21.15.27:0/2227222806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3b7004c7b0 con 0x7f3b80106b50 2023-12-17T12:01:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.731+0000 7f3b7e7fc700 1 --2- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f3b680387d0 0x7f3b6803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.731+0000 7f3b7e7fc700 1 --2- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f3b680387d0 0x7f3b6803ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b74006fd0 tx=0x7f3b74006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:44.735+0000 7f3b67fff700 1 -- 172.21.15.27:0/2227222806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3b7001b920 con 0x7f3b80106b50 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.002+0000 7f3b65ffb700 1 -- 172.21.15.27:0/2227222806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f3b60005160 con 0x7f3b80106b50 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.891+0000 7f3b67fff700 1 -- 172.21.15.27:0/2227222806 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f3b7000b3e0 con 0x7f3b80106b50 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.893+0000 7f3b67fff700 1 -- 172.21.15.27:0/2227222806 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f3b70023710 con 0x7f3b80106b50 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 -- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f3b680387d0 msgr2=0x7f3b6803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 --2- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f3b680387d0 0x7f3b6803ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b74006fd0 tx=0x7f3b74006e40 comp rx=0 tx=0).stop 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 -- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 msgr2=0x7f3b8019c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 --2- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b8019c9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b70003f20 tx=0x7f3b70011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 -- 172.21.15.27:0/2227222806 shutdown_connections 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 --2- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f3b680387d0 0x7f3b6803ac90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 --2- 172.21.15.27:0/2227222806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b80106b50 0x7f3b8019c9b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 -- 172.21.15.27:0/2227222806 >> 172.21.15.27:0/2227222806 conn(0x7f3b80102170 msgr2=0x7f3b80102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 -- 172.21.15.27:0/2227222806 shutdown_connections 2023-12-17T12:01:46.241 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:45.896+0000 7f3b65ffb700 1 -- 172.21.15.27:0/2227222806 wait complete. 2023-12-17T12:01:46.835 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:46 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:46.529+0000 7f244b092200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-17T12:01:46.836 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:46 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:46.659+0000 7f244b092200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-17T12:01:47.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:46 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:46.833+0000 7f244b092200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-17T12:01:47.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:46 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:46.949+0000 7f244b092200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-17T12:01:47.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:46 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2227222806' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-17T12:01:47.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:46 smithi027 ceph-mon[36871]: mgrmap e5: x(active, since 6s) 2023-12-17T12:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-17T12:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-17T12:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-17T12:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-17T12:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.656+0000 7f4b5ffff700 1 Processor -- start 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.656+0000 7f4b5ffff700 1 -- start start 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.656+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6007b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.656+0000 7f4b5ffff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6007b860 con 0x7f4b6007ce30 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.657+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6007b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.657+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6007b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44208/0 (socket says 172.21.15.27:44208) 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.657+0000 7f4b5effd700 1 -- 172.21.15.27:0/4092214727 learned_addr learned my addr 172.21.15.27:0/4092214727 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.657+0000 7f4b5effd700 1 -- 172.21.15.27:0/4092214727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6007b9a0 con 0x7f4b6007ce30 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.657+0000 7f4b5effd700 1 --2- 172.21.15.27:0/4092214727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6007b290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4b500098a0 tx=0x7f4b50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315727e6dc9d73af server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.658+0000 7f4b5dffb700 1 -- 172.21.15.27:0/4092214727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b50004160 con 0x7f4b6007ce30 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.658+0000 7f4b5dffb700 1 -- 172.21.15.27:0/4092214727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4b500042c0 con 0x7f4b6007ce30 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.658+0000 7f4b5dffb700 1 -- 172.21.15.27:0/4092214727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b50011940 con 0x7f4b6007ce30 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.659+0000 7f4b5ffff700 1 -- 172.21.15.27:0/4092214727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 msgr2=0x7f4b6007b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.659+0000 7f4b5ffff700 1 --2- 172.21.15.27:0/4092214727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6007b290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4b500098a0 tx=0x7f4b50009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.659+0000 7f4b5ffff700 1 -- 172.21.15.27:0/4092214727 shutdown_connections 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.659+0000 7f4b5ffff700 1 --2- 172.21.15.27:0/4092214727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6007b290 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.659+0000 7f4b5ffff700 1 -- 172.21.15.27:0/4092214727 >> 172.21.15.27:0/4092214727 conn(0x7f4b60101f90 msgr2=0x7f4b601043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.659+0000 7f4b5ffff700 1 -- 172.21.15.27:0/4092214727 shutdown_connections 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.659+0000 7f4b5ffff700 1 -- 172.21.15.27:0/4092214727 wait complete. 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.660+0000 7f4b5ffff700 1 Processor -- start 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.660+0000 7f4b5ffff700 1 -- start start 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.660+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.660+0000 7f4b5ffff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6019cd20 con 0x7f4b6007ce30 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.661+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.661+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44214/0 (socket says 172.21.15.27:44214) 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.661+0000 7f4b5effd700 1 -- 172.21.15.27:0/2665569150 learned_addr learned my addr 172.21.15.27:0/2665569150 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.661+0000 7f4b5effd700 1 -- 172.21.15.27:0/2665569150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50009550 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.661+0000 7f4b5effd700 1 --2- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6019c7e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4b50006af0 tx=0x7f4b50011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.662+0000 7f4b47fff700 1 -- 172.21.15.27:0/2665569150 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b50003bf0 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.662+0000 7f4b47fff700 1 -- 172.21.15.27:0/2665569150 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4b5001b8b0 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.662+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6019cf20 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.662+0000 7f4b47fff700 1 -- 172.21.15.27:0/2665569150 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b50023800 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.662+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6019d3c0 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.663+0000 7f4b47fff700 1 -- 172.21.15.27:0/2665569150 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f4b5001ba20 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.663+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6010cd00 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.663+0000 7f4b47fff700 1 --2- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f4b48038820 0x7f4b4803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.664+0000 7f4b47fff700 1 -- 172.21.15.27:0/2665569150 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4b5004ce50 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.664+0000 7f4b5e7fc700 1 -- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f4b48038820 msgr2=0x7f4b4803ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1943739335 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.664+0000 7f4b5e7fc700 1 --2- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f4b48038820 0x7f4b4803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.669+0000 7f4b47fff700 1 -- 172.21.15.27:0/2665569150 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4b5001bd70 con 0x7f4b6007ce30 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.865+0000 7f4b5e7fc700 1 -- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f4b48038820 msgr2=0x7f4b4803ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1943739335 2023-12-17T12:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:46.865+0000 7f4b5e7fc700 1 --2- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f4b48038820 0x7f4b4803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.000+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f4b6019fc50 con 0x7f4b6007ce30 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.000+0000 7f4b47fff700 1 -- 172.21.15.27:0/2665569150 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f4b5002b350 con 0x7f4b6007ce30 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f4b48038820 msgr2=0x7f4b4803ace0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 --2- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f4b48038820 0x7f4b4803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 msgr2=0x7f4b6019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 --2- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6019c7e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4b50006af0 tx=0x7f4b50011a50 comp rx=0 tx=0).stop 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 shutdown_connections 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 --2- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f4b48038820 0x7f4b4803ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 --2- 172.21.15.27:0/2665569150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b6007ce30 0x7f4b6019c7e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 >> 172.21.15.27:0/2665569150 conn(0x7f4b60101f90 msgr2=0x7f4b60102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 shutdown_connections 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.003+0000 7f4b5ffff700 1 -- 172.21.15.27:0/2665569150 wait complete. 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2023-12-17T12:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 5... 2023-12-17T12:01:48.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:47 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2665569150' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-17T12:01:48.161 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:47 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:47.836+0000 7f244b092200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:01:48.412 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:48.158+0000 7f244b092200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-17T12:01:48.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:48.274+0000 7f244b092200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-17T12:01:48.804 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:48.420+0000 7f244b092200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-17T12:01:48.805 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:48.623+0000 7f244b092200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-17T12:01:49.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:48.802+0000 7f244b092200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-17T12:01:49.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:48.919+0000 7f244b092200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-17T12:01:49.782 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:49 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:49.414+0000 7f244b092200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-17T12:01:50.150 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:49 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:49.779+0000 7f244b092200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-17T12:01:50.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:50 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:50.148+0000 7f244b092200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-17T12:01:50.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:50 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:50.276+0000 7f244b092200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-17T12:01:51.599 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:51 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:51.264+0000 7f244b092200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-17T12:01:51.599 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:51 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:51.394+0000 7f244b092200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-17T12:01:51.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:51 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:51.596+0000 7f244b092200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-17T12:01:52.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:52 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:52.453+0000 7f244b092200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-17T12:01:52.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:52 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:52.700+0000 7f244b092200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-17T12:01:53.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:52 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:52.965+0000 7f244b092200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-17T12:01:54.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:54.152+0000 7f244b092200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-17T12:01:54.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:54.523+0000 7f244b092200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:01:54.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:54.645+0000 7f244b092200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-17T12:01:55.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:01:54.910+0000 7f244b092200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-17T12:01:55.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: Active manager daemon x restarted 2023-12-17T12:01:55.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: Activating manager daemon x 2023-12-17T12:01:55.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: osdmap e2: 0 total, 0 up, 0 in 2023-12-17T12:01:55.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: mgrmap e6: x(active, starting, since 0.00359701s) 2023-12-17T12:01:55.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:01:55.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-17T12:01:55.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-17T12:01:55.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-17T12:01:55.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-17T12:01:55.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: Manager daemon x is now available 2023-12-17T12:01:55.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:01:55.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:54 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:01:56.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:01:56.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-17T12:01:56.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.814+0000 7f833a8f8700 1 Processor -- start 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.814+0000 7f833a8f8700 1 -- start start 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.815+0000 7f833a8f8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f8334106f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.815+0000 7f833a8f8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8334107530 con 0x7f8334106b40 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.815+0000 7f8333fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f8334106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.815+0000 7f8333fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f8334106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44228/0 (socket says 172.21.15.27:44228) 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.815+0000 7f8333fff700 1 -- 172.21.15.27:0/3067358674 learned_addr learned my addr 172.21.15.27:0/3067358674 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.815+0000 7f8333fff700 1 -- 172.21.15.27:0/3067358674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334107d40 con 0x7f8334106b40 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.816+0000 7f8333fff700 1 --2- 172.21.15.27:0/3067358674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f8334106f60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f83240098a0 tx=0x7f8324009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261493698962435b server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.816+0000 7f8332ffd700 1 -- 172.21.15.27:0/3067358674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8324004160 con 0x7f8334106b40 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.816+0000 7f8332ffd700 1 -- 172.21.15.27:0/3067358674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f83240042c0 con 0x7f8334106b40 2023-12-17T12:01:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.816+0000 7f8332ffd700 1 -- 172.21.15.27:0/3067358674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8324011940 con 0x7f8334106b40 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.816+0000 7f833a8f8700 1 -- 172.21.15.27:0/3067358674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 msgr2=0x7f8334106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.816+0000 7f833a8f8700 1 --2- 172.21.15.27:0/3067358674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f8334106f60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f83240098a0 tx=0x7f8324009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.817+0000 7f833a8f8700 1 -- 172.21.15.27:0/3067358674 shutdown_connections 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.817+0000 7f833a8f8700 1 --2- 172.21.15.27:0/3067358674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f8334106f60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.817+0000 7f833a8f8700 1 -- 172.21.15.27:0/3067358674 >> 172.21.15.27:0/3067358674 conn(0x7f8334102140 msgr2=0x7f8334104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.817+0000 7f833a8f8700 1 -- 172.21.15.27:0/3067358674 shutdown_connections 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.817+0000 7f833a8f8700 1 -- 172.21.15.27:0/3067358674 wait complete. 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.818+0000 7f833a8f8700 1 Processor -- start 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.818+0000 7f833a8f8700 1 -- start start 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.818+0000 7f833a8f8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f833419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.818+0000 7f833a8f8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833419cf60 con 0x7f8334106b40 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.818+0000 7f8333fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f833419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.819+0000 7f8333fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f833419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44230/0 (socket says 172.21.15.27:44230) 2023-12-17T12:01:56.293 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.819+0000 7f8333fff700 1 -- 172.21.15.27:0/1812135503 learned_addr learned my addr 172.21.15.27:0/1812135503 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.819+0000 7f8333fff700 1 -- 172.21.15.27:0/1812135503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324009550 con 0x7f8334106b40 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.819+0000 7f8333fff700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f833419ca20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8324003f20 tx=0x7f8324011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.820+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83240036a0 con 0x7f8334106b40 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.820+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f832401b440 con 0x7f8334106b40 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.820+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8324023400 con 0x7f8334106b40 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.820+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833419d160 con 0x7f8334106b40 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.820+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833419d600 con 0x7f8334106b40 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.821+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f8324003800 con 0x7f8334106b40 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.821+0000 7f83317fa700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.821+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 --> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f831c03b440 con 0x7f831c038870 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.821+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f832404c7d0 con 0x7f8334106b40 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.821+0000 7f83337fe700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 msgr2=0x7f831c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1943739335 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:47.821+0000 7f83337fe700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-17T12:01:56.294 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:48.022+0000 7f83337fe700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 msgr2=0x7f831c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1943739335 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:48.022+0000 7f83337fe700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:48.423+0000 7f83337fe700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 msgr2=0x7f831c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1943739335 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:48.423+0000 7f83337fe700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:49.224+0000 7f83337fe700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 msgr2=0x7f831c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1943739335 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:49.224+0000 7f83337fe700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:50.825+0000 7f83337fe700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 msgr2=0x7f831c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1943739335 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:50.826+0000 7f83337fe700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:54.029+0000 7f83337fe700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 msgr2=0x7f831c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1943739335 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:54.029+0000 7f83337fe700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:54.914+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 6) v1 ==== 45347+0+0 (secure 0 0 0) 0x7f8324023760 con 0x7f8334106b40 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:54.914+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 msgr2=0x7f831c03ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:54.914+0000 7f83317fa700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.915+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 7) v1 ==== 45474+0+0 (secure 0 0 0) 0x7f832404db20 con 0x7f8334106b40 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.916+0000 7f83317fa700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f831c03bfd0 0x7f831c03e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:56.295 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.916+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f831c03b440 con 0x7f831c03bfd0 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.916+0000 7f83337fe700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f831c03bfd0 0x7f831c03e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.918+0000 7f83337fe700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f831c03bfd0 0x7f831c03e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8328003a10 tx=0x7f83280092b0 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.919+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mgr.14122 v2:172.21.15.27:6800/674094332 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f831c03b440 con 0x7f831c03bfd0 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.926+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8334062c00 con 0x7f831c03bfd0 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.926+0000 7f83317fa700 1 -- 172.21.15.27:0/1812135503 <== mgr.14122 v2:172.21.15.27:6800/674094332 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8334062c00 con 0x7f831c03bfd0 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f831c03bfd0 msgr2=0x7f831c03e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f831c03bfd0 0x7f831c03e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8328003a10 tx=0x7f83280092b0 comp rx=0 tx=0).stop 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 msgr2=0x7f833419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f833419ca20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8324003f20 tx=0x7f8324011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 shutdown_connections 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f831c03bfd0 0x7f831c03e3c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334106b40 0x7f833419ca20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 --2- 172.21.15.27:0/1812135503 >> [v2:172.21.15.27:6800/1943739335,v1:172.21.15.27:6801/1943739335] conn(0x7f831c038870 0x7f831c03ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:56.296 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 >> 172.21.15.27:0/1812135503 conn(0x7f8334102140 msgr2=0x7f8334102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:56.297 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 shutdown_connections 2023-12-17T12:01:56.297 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:55.927+0000 7f833a8f8700 1 -- 172.21.15.27:0/1812135503 wait complete. 2023-12-17T12:01:56.297 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 5 is available 2023-12-17T12:01:56.297 INFO:teuthology.orchestra.run.smithi027.stdout:Setting orchestrator backend to cephadm... 2023-12-17T12:01:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: Found migration_current of "None". Setting to last migration. 2023-12-17T12:01:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:01:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:01:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:01:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-17T12:01:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-17T12:01:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:01:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:01:56.298 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:55 smithi027 ceph-mon[36871]: mgrmap e7: x(active, since 1.00617s) 2023-12-17T12:01:57.227 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:56 smithi027 ceph-mon[36871]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-17T12:01:57.227 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:56 smithi027 ceph-mon[36871]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-17T12:01:57.227 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:56 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:01:57.227 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:56 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:01:57.370 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.707+0000 7f3ab8328700 1 Processor -- start 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.707+0000 7f3ab8328700 1 -- start start 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.707+0000 7f3ab8328700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab007b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.707+0000 7f3ab8328700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab007b880 con 0x7f3ab007ce50 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.708+0000 7f3ab60c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab007b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.708+0000 7f3ab60c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab007b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42894/0 (socket says 172.21.15.27:42894) 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.708+0000 7f3ab60c4700 1 -- 172.21.15.27:0/3943864526 learned_addr learned my addr 172.21.15.27:0/3943864526 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.708+0000 7f3ab60c4700 1 -- 172.21.15.27:0/3943864526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab007b9c0 con 0x7f3ab007ce50 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.708+0000 7f3ab60c4700 1 --2- 172.21.15.27:0/3943864526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab007b2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0098a0 tx=0x7f3aac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db1e4b2fd750b09 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab50c2700 1 -- 172.21.15.27:0/3943864526 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3aac004160 con 0x7f3ab007ce50 2023-12-17T12:01:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab50c2700 1 -- 172.21.15.27:0/3943864526 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3aac0042c0 con 0x7f3ab007ce50 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab50c2700 1 -- 172.21.15.27:0/3943864526 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3aac011940 con 0x7f3ab007ce50 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab8328700 1 -- 172.21.15.27:0/3943864526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 msgr2=0x7f3ab007b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab8328700 1 --2- 172.21.15.27:0/3943864526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab007b2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0098a0 tx=0x7f3aac009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab8328700 1 -- 172.21.15.27:0/3943864526 shutdown_connections 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab8328700 1 --2- 172.21.15.27:0/3943864526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab007b2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab8328700 1 -- 172.21.15.27:0/3943864526 >> 172.21.15.27:0/3943864526 conn(0x7f3ab0102170 msgr2=0x7f3ab0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab8328700 1 -- 172.21.15.27:0/3943864526 shutdown_connections 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.709+0000 7f3ab8328700 1 -- 172.21.15.27:0/3943864526 wait complete. 2023-12-17T12:01:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.710+0000 7f3ab8328700 1 Processor -- start 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.711+0000 7f3ab8328700 1 -- start start 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.711+0000 7f3ab8328700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.711+0000 7f3ab8328700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab019cf80 con 0x7f3ab007ce50 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.711+0000 7f3ab60c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.711+0000 7f3ab60c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42904/0 (socket says 172.21.15.27:42904) 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.711+0000 7f3ab60c4700 1 -- 172.21.15.27:0/3377035598 learned_addr learned my addr 172.21.15.27:0/3377035598 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.712+0000 7f3ab60c4700 1 -- 172.21.15.27:0/3377035598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac009550 con 0x7f3ab007ce50 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.712+0000 7f3ab60c4700 1 --2- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab019ca40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3aac003f20 tx=0x7f3aac011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.712+0000 7f3aa77fe700 1 -- 172.21.15.27:0/3377035598 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3aac0036a0 con 0x7f3ab007ce50 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.712+0000 7f3aa77fe700 1 -- 172.21.15.27:0/3377035598 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3aac01b440 con 0x7f3ab007ce50 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.712+0000 7f3aa77fe700 1 -- 172.21.15.27:0/3377035598 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3aac023400 con 0x7f3ab007ce50 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.712+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab019d180 con 0x7f3ab007ce50 2023-12-17T12:01:57.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.712+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab019d620 con 0x7f3ab007ce50 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.714+0000 7f3aa77fe700 1 -- 172.21.15.27:0/3377035598 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) v1 ==== 45474+0+0 (secure 0 0 0) 0x7f3aac01b5b0 con 0x7f3ab007ce50 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.714+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab010cec0 con 0x7f3ab007ce50 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.714+0000 7f3aa77fe700 1 --2- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f3a9c038740 0x7f3a9c03ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.714+0000 7f3aa77fe700 1 -- 172.21.15.27:0/3377035598 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3aac04c620 con 0x7f3ab007ce50 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.714+0000 7f3ab58c3700 1 --2- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f3a9c038740 0x7f3a9c03ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.715+0000 7f3ab58c3700 1 --2- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f3a9c038740 0x7f3a9c03ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0006fd0 tx=0x7f3aa0006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.719+0000 7f3aa77fe700 1 -- 172.21.15.27:0/3377035598 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3aac01b920 con 0x7f3ab007ce50 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.945+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ab0103410 con 0x7f3a9c038740 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.952+0000 7f3aa77fe700 1 -- 172.21.15.27:0/3377035598 <== mgr.14122 v2:172.21.15.27:6800/674094332 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3ab0103410 con 0x7f3a9c038740 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.954+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f3a9c038740 msgr2=0x7f3a9c03ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.954+0000 7f3ab8328700 1 --2- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f3a9c038740 0x7f3a9c03ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0006fd0 tx=0x7f3aa0006e40 comp rx=0 tx=0).stop 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.954+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 msgr2=0x7f3ab019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.954+0000 7f3ab8328700 1 --2- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab019ca40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3aac003f20 tx=0x7f3aac011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:57.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.955+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 shutdown_connections 2023-12-17T12:01:57.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.955+0000 7f3ab8328700 1 --2- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f3a9c038740 0x7f3a9c03ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:57.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.955+0000 7f3ab8328700 1 --2- 172.21.15.27:0/3377035598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab007ce50 0x7f3ab019ca40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:57.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.955+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 >> 172.21.15.27:0/3377035598 conn(0x7f3ab0102170 msgr2=0x7f3ab0102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:57.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.955+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 shutdown_connections 2023-12-17T12:01:57.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:56.955+0000 7f3ab8328700 1 -- 172.21.15.27:0/3377035598 wait complete. 2023-12-17T12:01:58.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:57 smithi027 ceph-mon[36871]: [17/Dec/2023:12:01:56] ENGINE Bus STARTING 2023-12-17T12:01:58.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:57 smithi027 ceph-mon[36871]: [17/Dec/2023:12:01:56] ENGINE Serving on https://172.21.15.27:7150 2023-12-17T12:01:58.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:57 smithi027 ceph-mon[36871]: [17/Dec/2023:12:01:56] ENGINE Serving on http://172.21.15.27:8765 2023-12-17T12:01:58.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:57 smithi027 ceph-mon[36871]: [17/Dec/2023:12:01:56] ENGINE Bus STARTED 2023-12-17T12:01:58.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:57 smithi027 ceph-mon[36871]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:01:58.224 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:57 smithi027 ceph-mon[36871]: mgrmap e8: x(active, since 2s) 2023-12-17T12:01:58.225 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:57 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:01:58.338 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-17T12:01:58.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.779+0000 7f200f9bd700 1 Processor -- start 2023-12-17T12:01:58.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.779+0000 7f200f9bd700 1 -- start start 2023-12-17T12:01:58.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.780+0000 7f200f9bd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200807b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:58.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.780+0000 7f200f9bd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200807b880 con 0x7f200807ce50 2023-12-17T12:01:58.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.780+0000 7f200d759700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200807b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:58.343 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.780+0000 7f200d759700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200807b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42912/0 (socket says 172.21.15.27:42912) 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.780+0000 7f200d759700 1 -- 172.21.15.27:0/3928280967 learned_addr learned my addr 172.21.15.27:0/3928280967 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.781+0000 7f200d759700 1 -- 172.21.15.27:0/3928280967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200807b9c0 con 0x7f200807ce50 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.781+0000 7f200d759700 1 --2- 172.21.15.27:0/3928280967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200807b2b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20040098a0 tx=0x7f2004009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5340bfde330e66c server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.781+0000 7f1ffffff700 1 -- 172.21.15.27:0/3928280967 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2004004160 con 0x7f200807ce50 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.781+0000 7f1ffffff700 1 -- 172.21.15.27:0/3928280967 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f20040042c0 con 0x7f200807ce50 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.781+0000 7f1ffffff700 1 -- 172.21.15.27:0/3928280967 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2004011940 con 0x7f200807ce50 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.782+0000 7f200f9bd700 1 -- 172.21.15.27:0/3928280967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 msgr2=0x7f200807b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.782+0000 7f200f9bd700 1 --2- 172.21.15.27:0/3928280967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200807b2b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20040098a0 tx=0x7f2004009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.782+0000 7f200f9bd700 1 -- 172.21.15.27:0/3928280967 shutdown_connections 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.782+0000 7f200f9bd700 1 --2- 172.21.15.27:0/3928280967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200807b2b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.782+0000 7f200f9bd700 1 -- 172.21.15.27:0/3928280967 >> 172.21.15.27:0/3928280967 conn(0x7f2008102140 msgr2=0x7f2008104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.782+0000 7f200f9bd700 1 -- 172.21.15.27:0/3928280967 shutdown_connections 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.782+0000 7f200f9bd700 1 -- 172.21.15.27:0/3928280967 wait complete. 2023-12-17T12:01:58.344 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.783+0000 7f200f9bd700 1 Processor -- start 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.784+0000 7f200f9bd700 1 -- start start 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.784+0000 7f200f9bd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.784+0000 7f200f9bd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200819cf70 con 0x7f200807ce50 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.784+0000 7f200d759700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.784+0000 7f200d759700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42924/0 (socket says 172.21.15.27:42924) 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.784+0000 7f200d759700 1 -- 172.21.15.27:0/1236706041 learned_addr learned my addr 172.21.15.27:0/1236706041 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.785+0000 7f200d759700 1 -- 172.21.15.27:0/1236706041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2004009550 con 0x7f200807ce50 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.785+0000 7f200d759700 1 --2- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200819ca30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2004003f20 tx=0x7f2004011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.785+0000 7f1ffe7fc700 1 -- 172.21.15.27:0/1236706041 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20040036a0 con 0x7f200807ce50 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.785+0000 7f1ffe7fc700 1 -- 172.21.15.27:0/1236706041 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f200401b440 con 0x7f200807ce50 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.785+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200819d170 con 0x7f200807ce50 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.785+0000 7f1ffe7fc700 1 -- 172.21.15.27:0/1236706041 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2004023400 con 0x7f200807ce50 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.785+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200819d610 con 0x7f200807ce50 2023-12-17T12:01:58.345 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.786+0000 7f1ffe7fc700 1 -- 172.21.15.27:0/1236706041 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f200401b5b0 con 0x7f200807ce50 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.786+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200810ceb0 con 0x7f200807ce50 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.787+0000 7f1ffe7fc700 1 --2- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f1ff4038870 0x7f1ff403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.787+0000 7f1ffe7fc700 1 -- 172.21.15.27:0/1236706041 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f200404c8a0 con 0x7f200807ce50 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.787+0000 7f200cf58700 1 --2- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f1ff4038870 0x7f1ff403ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.787+0000 7f200cf58700 1 --2- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f1ff4038870 0x7f1ff403ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8006fd0 tx=0x7f1ff8006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:57.792+0000 7f1ffe7fc700 1 -- 172.21.15.27:0/1236706041 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f200401b920 con 0x7f200807ce50 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.004+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f20081033e0 con 0x7f1ff4038870 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.005+0000 7f1ffe7fc700 1 -- 172.21.15.27:0/1236706041 <== mgr.14122 v2:172.21.15.27:6800/674094332 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f20081033e0 con 0x7f1ff4038870 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f1ff4038870 msgr2=0x7f1ff403ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 --2- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f1ff4038870 0x7f1ff403ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8006fd0 tx=0x7f1ff8006e40 comp rx=0 tx=0).stop 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 msgr2=0x7f200819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 --2- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200819ca30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2004003f20 tx=0x7f2004011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 shutdown_connections 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 --2- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f1ff4038870 0x7f1ff403ad30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:58.346 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 --2- 172.21.15.27:0/1236706041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f200807ce50 0x7f200819ca30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:58.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 >> 172.21.15.27:0/1236706041 conn(0x7f2008102140 msgr2=0x7f2008102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:58.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 shutdown_connections 2023-12-17T12:01:58.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.008+0000 7f200f9bd700 1 -- 172.21.15.27:0/1236706041 wait complete. 2023-12-17T12:01:58.347 INFO:teuthology.orchestra.run.smithi027.stdout:Generating ssh key... 2023-12-17T12:01:59.295 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: Generating public/private rsa key pair. 2023-12-17T12:01:59.295 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: Your identification has been saved in /tmp/tmp0m_gj5q6/key. 2023-12-17T12:01:59.295 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: Your public key has been saved in /tmp/tmp0m_gj5q6/key.pub. 2023-12-17T12:01:59.295 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: The key fingerprint is: 2023-12-17T12:01:59.295 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: SHA256:Dldds9NC9UoZ5qGcO/xldfDJ4veWOjjBV6o3iBlZ37k ceph-62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:01:59.295 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: The key's randomart image is: 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: +---[RSA 3072]----+ 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | O..| 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | o B.Xo| 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | . =.Bo*| 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | . o.o.=o| 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | . S + =.=.+| 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | + o o *.=o| 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | . + * ..+| 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | o = +.E | 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: | o.o | 2023-12-17T12:01:59.296 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: +----[SHA256]-----+ 2023-12-17T12:01:59.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.745+0000 7fddb49f1700 1 Processor -- start 2023-12-17T12:01:59.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.746+0000 7fddb49f1700 1 -- start start 2023-12-17T12:01:59.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.746+0000 7fddb49f1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb007b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:59.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.746+0000 7fddb49f1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb007b8a0 con 0x7fddb007ce70 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.746+0000 7fddae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb007b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.746+0000 7fddae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb007b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42928/0 (socket says 172.21.15.27:42928) 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.746+0000 7fddae59c700 1 -- 172.21.15.27:0/678474221 learned_addr learned my addr 172.21.15.27:0/678474221 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.747+0000 7fddae59c700 1 -- 172.21.15.27:0/678474221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb007b9e0 con 0x7fddb007ce70 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.747+0000 7fddae59c700 1 --2- 172.21.15.27:0/678474221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb007b2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdda00098a0 tx=0x7fdda0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb754e3dd730b44 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.747+0000 7fddad59a700 1 -- 172.21.15.27:0/678474221 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdda0004160 con 0x7fddb007ce70 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.747+0000 7fddad59a700 1 -- 172.21.15.27:0/678474221 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fdda00042c0 con 0x7fddb007ce70 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.747+0000 7fddad59a700 1 -- 172.21.15.27:0/678474221 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdda0011940 con 0x7fddb007ce70 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.748+0000 7fddb49f1700 1 -- 172.21.15.27:0/678474221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 msgr2=0x7fddb007b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.748+0000 7fddb49f1700 1 --2- 172.21.15.27:0/678474221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb007b2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdda00098a0 tx=0x7fdda0009bb0 comp rx=0 tx=0).stop 2023-12-17T12:01:59.681 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.748+0000 7fddb49f1700 1 -- 172.21.15.27:0/678474221 shutdown_connections 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.748+0000 7fddb49f1700 1 --2- 172.21.15.27:0/678474221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb007b2d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.748+0000 7fddb49f1700 1 -- 172.21.15.27:0/678474221 >> 172.21.15.27:0/678474221 conn(0x7fddb0102140 msgr2=0x7fddb0104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.748+0000 7fddb49f1700 1 -- 172.21.15.27:0/678474221 shutdown_connections 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.748+0000 7fddb49f1700 1 -- 172.21.15.27:0/678474221 wait complete. 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.749+0000 7fddb49f1700 1 Processor -- start 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.749+0000 7fddb49f1700 1 -- start start 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.749+0000 7fddb49f1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.749+0000 7fddb49f1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb019cf00 con 0x7fddb007ce70 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fddae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fddae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42936/0 (socket says 172.21.15.27:42936) 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fddae59c700 1 -- 172.21.15.27:0/3873364633 learned_addr learned my addr 172.21.15.27:0/3873364633 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fddae59c700 1 -- 172.21.15.27:0/3873364633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda0009550 con 0x7fddb007ce70 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fddae59c700 1 --2- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb019c9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdda0003f20 tx=0x7fdda0011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:59.682 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fdd9f7fe700 1 -- 172.21.15.27:0/3873364633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdda00036a0 con 0x7fddb007ce70 2023-12-17T12:01:59.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fdd9f7fe700 1 -- 172.21.15.27:0/3873364633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fdda001b440 con 0x7fddb007ce70 2023-12-17T12:01:59.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fdd9f7fe700 1 -- 172.21.15.27:0/3873364633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdda0023400 con 0x7fddb007ce70 2023-12-17T12:01:59.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.750+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb019d100 con 0x7fddb007ce70 2023-12-17T12:01:59.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.751+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb019d5a0 con 0x7fddb007ce70 2023-12-17T12:01:59.683 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.752+0000 7fdd9f7fe700 1 -- 172.21.15.27:0/3873364633 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fdda001b5b0 con 0x7fddb007ce70 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.752+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb010ce90 con 0x7fddb007ce70 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.752+0000 7fdd9f7fe700 1 --2- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7fdd98038870 0x7fdd9803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.752+0000 7fdd9f7fe700 1 -- 172.21.15.27:0/3873364633 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdda004c7a0 con 0x7fddb007ce70 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.753+0000 7fddadd9b700 1 --2- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7fdd98038870 0x7fdd9803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.753+0000 7fddadd9b700 1 --2- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7fdd98038870 0x7fdd9803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdda4006fd0 tx=0x7fdda4006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.758+0000 7fdd9f7fe700 1 -- 172.21.15.27:0/3873364633 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdda001b920 con 0x7fddb007ce70 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:58.970+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fddb01033c0 con 0x7fdd98038870 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.257+0000 7fdd9f7fe700 1 -- 172.21.15.27:0/3873364633 <== mgr.14122 v2:172.21.15.27:6800/674094332 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fddb01033c0 con 0x7fdd98038870 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.260+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7fdd98038870 msgr2=0x7fdd9803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.260+0000 7fddb49f1700 1 --2- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7fdd98038870 0x7fdd9803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdda4006fd0 tx=0x7fdda4006e40 comp rx=0 tx=0).stop 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.260+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 msgr2=0x7fddb019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.260+0000 7fddb49f1700 1 --2- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb019c9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdda0003f20 tx=0x7fdda0011f00 comp rx=0 tx=0).stop 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.261+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 shutdown_connections 2023-12-17T12:01:59.684 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.261+0000 7fddb49f1700 1 --2- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7fdd98038870 0x7fdd9803ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:59.685 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.261+0000 7fddb49f1700 1 --2- 172.21.15.27:0/3873364633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb007ce70 0x7fddb019c9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:01:59.685 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.261+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 >> 172.21.15.27:0/3873364633 conn(0x7fddb0102140 msgr2=0x7fddb0102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:01:59.685 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.261+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 shutdown_connections 2023-12-17T12:01:59.685 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:01:59.261+0000 7fddb49f1700 1 -- 172.21.15.27:0/3873364633 wait complete. 2023-12-17T12:01:59.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-mon[36871]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:01:59.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-mon[36871]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:01:59.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-mon[36871]: Generating ssh key... 2023-12-17T12:01:59.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:01:59.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:01:59 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:02:00.826 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf6OmOd2AJDKBzcKyeXgHHHTikb/7kSocblHmTbIJuY9Vvts6Ri8Tafg7VzwP3W+WGF0v1QBqwZtKZH6EvSjlIqSu3/Fh9yIrgOeRXadfVNh0aiWlBCWNzs4FeedEVJcFKDE4PtMkyZxV6FxeS2pZ8Tzd270FO4JlfxWCwQMMEEfPReZdEdZkygTyGSCDeBGkVpRxl5ovFok4wZsGqRuijO+tYr0Ca92cO4aAZEoaIYYgbneNT+FvBr8t+/IzNoUBiedfnc/OdM0u+uPgVKxzkere9IxkVNxgcUNSeNJEeZDw8/gxGdr4qEl6xgNekERB+1S+E5eV4Rf5mN1An7ZKQRNMOwLuubwVaqijXsh9gtBIGwQyaNFSBQ2maj0B42oywoLxJhDXYgwaDgjoNPz2pmzPYN4sy+RkNTx84WpemtWlSjJ/0pkOqih0za1TM5FAyNv60IWd50WUHJc/lsYyDXo89qTGORqCHf3zyBjw0U0XhxD7ZCLR6u8YLQBNMBE0= ceph-62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.145+0000 7f7dde450700 1 Processor -- start 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.145+0000 7f7dde450700 1 -- start start 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.146+0000 7f7dde450700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd807b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.146+0000 7f7dde450700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd807b860 con 0x7f7dd807ce30 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.146+0000 7f7dd7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd807b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.146+0000 7f7dd7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd807b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42940/0 (socket says 172.21.15.27:42940) 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.146+0000 7f7dd7fff700 1 -- 172.21.15.27:0/1803506069 learned_addr learned my addr 172.21.15.27:0/1803506069 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.147+0000 7f7dd7fff700 1 -- 172.21.15.27:0/1803506069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd807b9a0 con 0x7f7dd807ce30 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.147+0000 7f7dd7fff700 1 --2- 172.21.15.27:0/1803506069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd807b290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80098a0 tx=0x7f7dc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1983f96768c66a3e server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.147+0000 7f7dd6ffd700 1 -- 172.21.15.27:0/1803506069 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dc8004160 con 0x7f7dd807ce30 2023-12-17T12:02:00.827 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.147+0000 7f7dd6ffd700 1 -- 172.21.15.27:0/1803506069 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f7dc80042c0 con 0x7f7dd807ce30 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.147+0000 7f7dd6ffd700 1 -- 172.21.15.27:0/1803506069 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dc8011940 con 0x7f7dd807ce30 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.148+0000 7f7dde450700 1 -- 172.21.15.27:0/1803506069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 msgr2=0x7f7dd807b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.148+0000 7f7dde450700 1 --2- 172.21.15.27:0/1803506069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd807b290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80098a0 tx=0x7f7dc8009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.148+0000 7f7dde450700 1 -- 172.21.15.27:0/1803506069 shutdown_connections 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.148+0000 7f7dde450700 1 --2- 172.21.15.27:0/1803506069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd807b290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.148+0000 7f7dde450700 1 -- 172.21.15.27:0/1803506069 >> 172.21.15.27:0/1803506069 conn(0x7f7dd8102140 msgr2=0x7f7dd8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.148+0000 7f7dde450700 1 -- 172.21.15.27:0/1803506069 shutdown_connections 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.148+0000 7f7dde450700 1 -- 172.21.15.27:0/1803506069 wait complete. 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.149+0000 7f7dde450700 1 Processor -- start 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.149+0000 7f7dde450700 1 -- start start 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.150+0000 7f7dde450700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.150+0000 7f7dde450700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd819cea0 con 0x7f7dd807ce30 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.150+0000 7f7dd7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.150+0000 7f7dd7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd819c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42956/0 (socket says 172.21.15.27:42956) 2023-12-17T12:02:00.828 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.150+0000 7f7dd7fff700 1 -- 172.21.15.27:0/3798830542 learned_addr learned my addr 172.21.15.27:0/3798830542 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.150+0000 7f7dd7fff700 1 -- 172.21.15.27:0/3798830542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8009550 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.151+0000 7f7dd7fff700 1 --2- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd819c960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8003f20 tx=0x7f7dc8011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.151+0000 7f7dd57fa700 1 -- 172.21.15.27:0/3798830542 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dc80037d0 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.151+0000 7f7dd57fa700 1 -- 172.21.15.27:0/3798830542 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f7dc801b440 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.151+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd819d0a0 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.151+0000 7f7dd57fa700 1 -- 172.21.15.27:0/3798830542 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dc8023590 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.151+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd819d540 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.152+0000 7f7dd57fa700 1 -- 172.21.15.27:0/3798830542 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f7dc8023800 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.152+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd810ced0 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.153+0000 7f7dd57fa700 1 --2- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f7dc0038870 0x7f7dc003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.153+0000 7f7dd57fa700 1 -- 172.21.15.27:0/3798830542 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7dc804ca10 con 0x7f7dd807ce30 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.153+0000 7f7dd77fe700 1 --2- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f7dc0038870 0x7f7dc003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:00.829 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.154+0000 7f7dd77fe700 1 --2- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f7dc0038870 0x7f7dc003ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc006fd0 tx=0x7f7dcc006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.158+0000 7f7dd57fa700 1 -- 172.21.15.27:0/3798830542 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7dc8023ab0 con 0x7f7dd807ce30 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.373+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dd81033c0 con 0x7f7dc0038870 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.374+0000 7f7dd57fa700 1 -- 172.21.15.27:0/3798830542 <== mgr.14122 v2:172.21.15.27:6800/674094332 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f7dd81033c0 con 0x7f7dc0038870 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f7dc0038870 msgr2=0x7f7dc003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 --2- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f7dc0038870 0x7f7dc003ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc006fd0 tx=0x7f7dcc006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 msgr2=0x7f7dd819c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 --2- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd819c960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8003f20 tx=0x7f7dc8011f00 comp rx=0 tx=0).stop 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 shutdown_connections 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 --2- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f7dc0038870 0x7f7dc003ad30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 --2- 172.21.15.27:0/3798830542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dd807ce30 0x7f7dd819c960 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 >> 172.21.15.27:0/3798830542 conn(0x7f7dd8102140 msgr2=0x7f7dd8102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 shutdown_connections 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:00.377+0000 7f7dde450700 1 -- 172.21.15.27:0/3798830542 wait complete. 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-17T12:02:00.830 INFO:teuthology.orchestra.run.smithi027.stdout:Adding key to root@localhost authorized_keys... 2023-12-17T12:02:00.831 INFO:teuthology.orchestra.run.smithi027.stdout:Adding host smithi027... 2023-12-17T12:02:01.406 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:01 smithi027 ceph-mon[36871]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:02.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:02 smithi027 ceph-mon[36871]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:03 smithi027 ceph-mon[36871]: Deploying cephadm binary to smithi027 2023-12-17T12:02:06.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Added host 'smithi027' with addr '172.21.15.27' 2023-12-17T12:02:06.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.310+0000 7f811585a700 1 Processor -- start 2023-12-17T12:02:06.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.310+0000 7f811585a700 1 -- start start 2023-12-17T12:02:06.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.310+0000 7f811585a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f8110109a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:06.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.310+0000 7f811585a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811007ae00 con 0x7f8110107620 2023-12-17T12:02:06.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.311+0000 7f810effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f8110109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:06.895 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.311+0000 7f810effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f8110109a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42968/0 (socket says 172.21.15.27:42968) 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.311+0000 7f810effd700 1 -- 172.21.15.27:0/3223944363 learned_addr learned my addr 172.21.15.27:0/3223944363 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.311+0000 7f810effd700 1 -- 172.21.15.27:0/3223944363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110109f50 con 0x7f8110107620 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.312+0000 7f810effd700 1 --2- 172.21.15.27:0/3223944363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f8110109a10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f81000098a0 tx=0x7f8100009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c0f03efcb65522 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.312+0000 7f810dffb700 1 -- 172.21.15.27:0/3223944363 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8100004160 con 0x7f8110107620 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.312+0000 7f810dffb700 1 -- 172.21.15.27:0/3223944363 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f81000042c0 con 0x7f8110107620 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.312+0000 7f810dffb700 1 -- 172.21.15.27:0/3223944363 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8100011940 con 0x7f8110107620 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.313+0000 7f811585a700 1 -- 172.21.15.27:0/3223944363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 msgr2=0x7f8110109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.313+0000 7f811585a700 1 --2- 172.21.15.27:0/3223944363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f8110109a10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f81000098a0 tx=0x7f8100009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.313+0000 7f811585a700 1 -- 172.21.15.27:0/3223944363 shutdown_connections 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.313+0000 7f811585a700 1 --2- 172.21.15.27:0/3223944363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f8110109a10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.313+0000 7f811585a700 1 -- 172.21.15.27:0/3223944363 >> 172.21.15.27:0/3223944363 conn(0x7f8110101030 msgr2=0x7f8110103450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.313+0000 7f811585a700 1 -- 172.21.15.27:0/3223944363 shutdown_connections 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.313+0000 7f811585a700 1 -- 172.21.15.27:0/3223944363 wait complete. 2023-12-17T12:02:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.314+0000 7f811585a700 1 Processor -- start 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.314+0000 7f811585a700 1 -- start start 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.314+0000 7f811585a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f811019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.315+0000 7f811585a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811019cf40 con 0x7f8110107620 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.315+0000 7f810effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f811019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.315+0000 7f810effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f811019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42972/0 (socket says 172.21.15.27:42972) 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.315+0000 7f810effd700 1 -- 172.21.15.27:0/1786314733 learned_addr learned my addr 172.21.15.27:0/1786314733 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.315+0000 7f810effd700 1 -- 172.21.15.27:0/1786314733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8100009550 con 0x7f8110107620 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.315+0000 7f810effd700 1 --2- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f811019ca00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8100003f20 tx=0x7f8100011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.316+0000 7f8114858700 1 -- 172.21.15.27:0/1786314733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81000037d0 con 0x7f8110107620 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.316+0000 7f8114858700 1 -- 172.21.15.27:0/1786314733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f810001b440 con 0x7f8110107620 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.316+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811019d140 con 0x7f8110107620 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.316+0000 7f8114858700 1 -- 172.21.15.27:0/1786314733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8100023590 con 0x7f8110107620 2023-12-17T12:02:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.316+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811019d5e0 con 0x7f8110107620 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.317+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811010ac90 con 0x7f8110107620 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.318+0000 7f8114858700 1 -- 172.21.15.27:0/1786314733 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f8100023800 con 0x7f8110107620 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.318+0000 7f8114858700 1 --2- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f80f8038870 0x7f80f803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.318+0000 7f8114858700 1 -- 172.21.15.27:0/1786314733 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f810004c8e0 con 0x7f8110107620 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.319+0000 7f810e7fc700 1 --2- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f80f8038870 0x7f80f803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.319+0000 7f810e7fc700 1 --2- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f80f8038870 0x7f80f803ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8104006fd0 tx=0x7f8104006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.323+0000 7f8114858700 1 -- 172.21.15.27:0/1786314733 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8100023ab0 con 0x7f8110107620 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:01.542+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}) v1 -- 0x7f81101022c0 con 0x7f80f8038870 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.564+0000 7f8114858700 1 -- 172.21.15.27:0/1786314733 <== mgr.14122 v2:172.21.15.27:6800/674094332 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f81101022c0 con 0x7f80f8038870 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f80f8038870 msgr2=0x7f80f803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 --2- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f80f8038870 0x7f80f803ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8104006fd0 tx=0x7f8104006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 msgr2=0x7f811019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 --2- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f811019ca00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8100003f20 tx=0x7f8100011f00 comp rx=0 tx=0).stop 2023-12-17T12:02:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 shutdown_connections 2023-12-17T12:02:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 --2- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f80f8038870 0x7f80f803ad30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 --2- 172.21.15.27:0/1786314733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8110107620 0x7f811019ca00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 >> 172.21.15.27:0/1786314733 conn(0x7f8110101030 msgr2=0x7f8110101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.567+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 shutdown_connections 2023-12-17T12:02:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:06.568+0000 7f811585a700 1 -- 172.21.15.27:0/1786314733 wait complete. 2023-12-17T12:02:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mon service... 2023-12-17T12:02:07.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:07 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:02:07.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:07 smithi027 ceph-mon[36871]: Added host smithi027 2023-12-17T12:02:07.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:07 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:02:08.063 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-17T12:02:08.063 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.343+0000 7f45ddec4700 1 Processor -- start 2023-12-17T12:02:08.063 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.343+0000 7f45ddec4700 1 -- start start 2023-12-17T12:02:08.063 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.343+0000 7f45ddec4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:08.063 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.343+0000 7f45ddec4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8107540 con 0x7f45d8106b50 2023-12-17T12:02:08.063 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.344+0000 7f45d77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.344+0000 7f45d77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d8106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43624/0 (socket says 172.21.15.27:43624) 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.344+0000 7f45d77fe700 1 -- 172.21.15.27:0/1676238881 learned_addr learned my addr 172.21.15.27:0/1676238881 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.344+0000 7f45d77fe700 1 -- 172.21.15.27:0/1676238881 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d8107d50 con 0x7f45d8106b50 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.345+0000 7f45d77fe700 1 --2- 172.21.15.27:0/1676238881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d8106f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45c80098a0 tx=0x7f45c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d78dd229e429f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.345+0000 7f45d67fc700 1 -- 172.21.15.27:0/1676238881 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c8004160 con 0x7f45d8106b50 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.345+0000 7f45d67fc700 1 -- 172.21.15.27:0/1676238881 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f45c80042c0 con 0x7f45d8106b50 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.345+0000 7f45d67fc700 1 -- 172.21.15.27:0/1676238881 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c8011940 con 0x7f45d8106b50 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.345+0000 7f45ddec4700 1 -- 172.21.15.27:0/1676238881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 msgr2=0x7f45d8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.345+0000 7f45ddec4700 1 --2- 172.21.15.27:0/1676238881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d8106f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45c80098a0 tx=0x7f45c8009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.346+0000 7f45ddec4700 1 -- 172.21.15.27:0/1676238881 shutdown_connections 2023-12-17T12:02:08.064 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.346+0000 7f45ddec4700 1 --2- 172.21.15.27:0/1676238881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d8106f70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:08.068 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.346+0000 7f45ddec4700 1 -- 172.21.15.27:0/1676238881 >> 172.21.15.27:0/1676238881 conn(0x7f45d8102170 msgr2=0x7f45d8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:08.068 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.346+0000 7f45ddec4700 1 -- 172.21.15.27:0/1676238881 shutdown_connections 2023-12-17T12:02:08.068 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.346+0000 7f45ddec4700 1 -- 172.21.15.27:0/1676238881 wait complete. 2023-12-17T12:02:08.068 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.347+0000 7f45ddec4700 1 Processor -- start 2023-12-17T12:02:08.068 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.347+0000 7f45ddec4700 1 -- start start 2023-12-17T12:02:08.068 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.347+0000 7f45ddec4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d807cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:08.068 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.347+0000 7f45ddec4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d807d010 con 0x7f45d8106b50 2023-12-17T12:02:08.068 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45d77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d807cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45d77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d807cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43636/0 (socket says 172.21.15.27:43636) 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45d77fe700 1 -- 172.21.15.27:0/2143707608 learned_addr learned my addr 172.21.15.27:0/2143707608 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45d77fe700 1 -- 172.21.15.27:0/2143707608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c8009550 con 0x7f45d8106b50 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45d77fe700 1 --2- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d807cad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f45c8003f20 tx=0x7f45c8011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45d4ff9700 1 -- 172.21.15.27:0/2143707608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c80036a0 con 0x7f45d8106b50 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45d4ff9700 1 -- 172.21.15.27:0/2143707608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f45c801b440 con 0x7f45d8106b50 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45d4ff9700 1 -- 172.21.15.27:0/2143707608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c8023400 con 0x7f45d8106b50 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.348+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d807b250 con 0x7f45d8106b50 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.349+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d807b6f0 con 0x7f45d8106b50 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.350+0000 7f45d4ff9700 1 -- 172.21.15.27:0/2143707608 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f45c8003800 con 0x7f45d8106b50 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.350+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8062c00 con 0x7f45d8106b50 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.350+0000 7f45d4ff9700 1 --2- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f45c0038870 0x7f45c003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.350+0000 7f45d4ff9700 1 -- 172.21.15.27:0/2143707608 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f45c804c8a0 con 0x7f45d8106b50 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.351+0000 7f45d6ffd700 1 --2- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f45c0038870 0x7f45c003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.351+0000 7f45d6ffd700 1 --2- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f45c0038870 0x7f45c003ad30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f45cc006fd0 tx=0x7f45cc006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.356+0000 7f45d4ff9700 1 -- 172.21.15.27:0/2143707608 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f45c800ba30 con 0x7f45d8106b50 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.605+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f45d807be50 con 0x7f45c0038870 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.615+0000 7f45d4ff9700 1 -- 172.21.15.27:0/2143707608 <== mgr.14122 v2:172.21.15.27:6800/674094332 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f45d807be50 con 0x7f45c0038870 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f45c0038870 msgr2=0x7f45c003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 --2- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f45c0038870 0x7f45c003ad30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f45cc006fd0 tx=0x7f45cc006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 msgr2=0x7f45d807cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 --2- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d807cad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f45c8003f20 tx=0x7f45c8011f00 comp rx=0 tx=0).stop 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 shutdown_connections 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 --2- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f45c0038870 0x7f45c003ad30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 --2- 172.21.15.27:0/2143707608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8106b50 0x7f45d807cad0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 >> 172.21.15.27:0/2143707608 conn(0x7f45d8102170 msgr2=0x7f45d8102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:08.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 shutdown_connections 2023-12-17T12:02:08.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:07.618+0000 7f45ddec4700 1 -- 172.21.15.27:0/2143707608 wait complete. 2023-12-17T12:02:08.071 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mgr service... 2023-12-17T12:02:08.893 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:08 smithi027 ceph-mon[36871]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:08.894 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:08 smithi027 ceph-mon[36871]: Saving service mon spec with placement count:5 2023-12-17T12:02:08.894 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:08 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:02:09.271 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-17T12:02:09.271 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.616+0000 7f641955c700 1 Processor -- start 2023-12-17T12:02:09.271 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.616+0000 7f641955c700 1 -- start start 2023-12-17T12:02:09.271 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.617+0000 7f641955c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f641407dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:09.271 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.617+0000 7f641955c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641407e200 con 0x7f641407d810 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.617+0000 7f6412ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f641407dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.617+0000 7f6412ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f641407dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43650/0 (socket says 172.21.15.27:43650) 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.617+0000 7f6412ffd700 1 -- 172.21.15.27:0/3310504345 learned_addr learned my addr 172.21.15.27:0/3310504345 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.618+0000 7f6412ffd700 1 -- 172.21.15.27:0/3310504345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641407ea60 con 0x7f641407d810 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.618+0000 7f6412ffd700 1 --2- 172.21.15.27:0/3310504345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f641407dc30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64040098a0 tx=0x7f6404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503bf794e10828f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.618+0000 7f6411ffb700 1 -- 172.21.15.27:0/3310504345 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6404004160 con 0x7f641407d810 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.618+0000 7f6411ffb700 1 -- 172.21.15.27:0/3310504345 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f64040042c0 con 0x7f641407d810 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.618+0000 7f6411ffb700 1 -- 172.21.15.27:0/3310504345 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6404011940 con 0x7f641407d810 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.619+0000 7f641955c700 1 -- 172.21.15.27:0/3310504345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 msgr2=0x7f641407dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.619+0000 7f641955c700 1 --2- 172.21.15.27:0/3310504345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f641407dc30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64040098a0 tx=0x7f6404009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.619+0000 7f641955c700 1 -- 172.21.15.27:0/3310504345 shutdown_connections 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.619+0000 7f641955c700 1 --2- 172.21.15.27:0/3310504345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f641407dc30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:09.272 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.619+0000 7f641955c700 1 -- 172.21.15.27:0/3310504345 >> 172.21.15.27:0/3310504345 conn(0x7f641407c150 msgr2=0x7f641407c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.619+0000 7f641955c700 1 -- 172.21.15.27:0/3310504345 shutdown_connections 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.619+0000 7f641955c700 1 -- 172.21.15.27:0/3310504345 wait complete. 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.621+0000 7f641955c700 1 Processor -- start 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.621+0000 7f641955c700 1 -- start start 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.621+0000 7f641955c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f64141a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.622+0000 7f641955c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141a1320 con 0x7f641407d810 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.622+0000 7f6412ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f64141a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.622+0000 7f6412ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f64141a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43662/0 (socket says 172.21.15.27:43662) 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.622+0000 7f6412ffd700 1 -- 172.21.15.27:0/4123434670 learned_addr learned my addr 172.21.15.27:0/4123434670 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.622+0000 7f6412ffd700 1 -- 172.21.15.27:0/4123434670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6404009550 con 0x7f641407d810 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.622+0000 7f6412ffd700 1 --2- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f64141a0de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6404003f20 tx=0x7f6404011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.623+0000 7f63fbfff700 1 -- 172.21.15.27:0/4123434670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64040037d0 con 0x7f641407d810 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.623+0000 7f63fbfff700 1 -- 172.21.15.27:0/4123434670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f640401b440 con 0x7f641407d810 2023-12-17T12:02:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.623+0000 7f63fbfff700 1 -- 172.21.15.27:0/4123434670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6404023590 con 0x7f641407d810 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.623+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64141a1520 con 0x7f641407d810 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.623+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64141a19c0 con 0x7f641407d810 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.624+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641419ab70 con 0x7f641407d810 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.624+0000 7f63fbfff700 1 -- 172.21.15.27:0/4123434670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f640401b5b0 con 0x7f641407d810 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.625+0000 7f63fbfff700 1 --2- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f63fc038870 0x7f63fc03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.625+0000 7f63fbfff700 1 -- 172.21.15.27:0/4123434670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f640404c840 con 0x7f641407d810 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.625+0000 7f64127fc700 1 --2- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f63fc038870 0x7f63fc03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.626+0000 7f64127fc700 1 --2- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f63fc038870 0x7f63fc03ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6408006fd0 tx=0x7f6408006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.630+0000 7f63fbfff700 1 -- 172.21.15.27:0/4123434670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f640401b920 con 0x7f641407d810 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.666+0000 7f63fbfff700 1 -- 172.21.15.27:0/4123434670 <== mon.0 v2:172.21.15.27:3300/0 7 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6404023e80 con 0x7f641407d810 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.861+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6414061a10 con 0x7f63fc038870 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.867+0000 7f63fbfff700 1 -- 172.21.15.27:0/4123434670 <== mgr.14122 v2:172.21.15.27:6800/674094332 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6414061a10 con 0x7f63fc038870 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.869+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f63fc038870 msgr2=0x7f63fc03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.869+0000 7f641955c700 1 --2- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f63fc038870 0x7f63fc03ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6408006fd0 tx=0x7f6408006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:09.274 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.869+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 msgr2=0x7f64141a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:09.275 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.869+0000 7f641955c700 1 --2- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f64141a0de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6404003f20 tx=0x7f6404011f00 comp rx=0 tx=0).stop 2023-12-17T12:02:09.275 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.869+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 shutdown_connections 2023-12-17T12:02:09.275 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.869+0000 7f641955c700 1 --2- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f63fc038870 0x7f63fc03ad30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:09.275 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.869+0000 7f641955c700 1 --2- 172.21.15.27:0/4123434670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641407d810 0x7f64141a0de0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:09.275 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.869+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 >> 172.21.15.27:0/4123434670 conn(0x7f641407c150 msgr2=0x7f6414197cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:09.275 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.870+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 shutdown_connections 2023-12-17T12:02:09.275 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:08.870+0000 7f641955c700 1 -- 172.21.15.27:0/4123434670 wait complete. 2023-12-17T12:02:09.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:09 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:02:09.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:09 smithi027 ceph-mon[36871]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:09.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:09 smithi027 ceph-mon[36871]: Saving service mgr spec with placement count:2 2023-12-17T12:02:09.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:09 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:02:09.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:09 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:02:10.702 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.832+0000 7f30149b0700 1 Processor -- start 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.832+0000 7f30149b0700 1 -- start start 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.832+0000 7f30149b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301007b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.832+0000 7f30149b0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301007b880 con 0x7f301007ce50 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.833+0000 7f300e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301007b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.833+0000 7f300e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301007b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43674/0 (socket says 172.21.15.27:43674) 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.833+0000 7f300e59c700 1 -- 172.21.15.27:0/4225788311 learned_addr learned my addr 172.21.15.27:0/4225788311 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.833+0000 7f300e59c700 1 -- 172.21.15.27:0/4225788311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301007b9c0 con 0x7f301007ce50 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.833+0000 7f300e59c700 1 --2- 172.21.15.27:0/4225788311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301007b2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30000098a0 tx=0x7f3000009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9856a8d1e172aa82 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.834+0000 7f300d59a700 1 -- 172.21.15.27:0/4225788311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3000004160 con 0x7f301007ce50 2023-12-17T12:02:10.703 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.834+0000 7f300d59a700 1 -- 172.21.15.27:0/4225788311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30000042c0 con 0x7f301007ce50 2023-12-17T12:02:10.707 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.834+0000 7f300d59a700 1 -- 172.21.15.27:0/4225788311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30000118e0 con 0x7f301007ce50 2023-12-17T12:02:10.707 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.834+0000 7f30149b0700 1 -- 172.21.15.27:0/4225788311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 msgr2=0x7f301007b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:10.707 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.834+0000 7f30149b0700 1 --2- 172.21.15.27:0/4225788311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301007b2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30000098a0 tx=0x7f3000009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:10.707 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.834+0000 7f30149b0700 1 -- 172.21.15.27:0/4225788311 shutdown_connections 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.834+0000 7f30149b0700 1 --2- 172.21.15.27:0/4225788311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301007b2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.834+0000 7f30149b0700 1 -- 172.21.15.27:0/4225788311 >> 172.21.15.27:0/4225788311 conn(0x7f3010102140 msgr2=0x7f3010104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.835+0000 7f30149b0700 1 -- 172.21.15.27:0/4225788311 shutdown_connections 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.835+0000 7f30149b0700 1 -- 172.21.15.27:0/4225788311 wait complete. 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.836+0000 7f30149b0700 1 Processor -- start 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.836+0000 7f30149b0700 1 -- start start 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.836+0000 7f30149b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.836+0000 7f30149b0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301019cfe0 con 0x7f301007ce50 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.837+0000 7f300e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.837+0000 7f300e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43680/0 (socket says 172.21.15.27:43680) 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.837+0000 7f300e59c700 1 -- 172.21.15.27:0/2314920128 learned_addr learned my addr 172.21.15.27:0/2314920128 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.837+0000 7f300e59c700 1 -- 172.21.15.27:0/2314920128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009550 con 0x7f301007ce50 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.837+0000 7f300e59c700 1 --2- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301019caa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3000003f70 tx=0x7f30000119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:10.708 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.837+0000 7f2fff7fe700 1 -- 172.21.15.27:0/2314920128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30000036a0 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.837+0000 7f2fff7fe700 1 -- 172.21.15.27:0/2314920128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f300001b440 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.838+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301019d1e0 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.838+0000 7f2fff7fe700 1 -- 172.21.15.27:0/2314920128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3000023400 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.838+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301019d680 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.839+0000 7f2fff7fe700 1 -- 172.21.15.27:0/2314920128 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f3000023630 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.840+0000 7f2fff7fe700 1 --2- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f2ff8038870 0x7f2ff803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.840+0000 7f2fff7fe700 1 -- 172.21.15.27:0/2314920128 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3000052580 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.840+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301010cf00 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.840+0000 7f300dd9b700 1 --2- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f2ff8038870 0x7f2ff803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.841+0000 7f300dd9b700 1 --2- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f2ff8038870 0x7f2ff803ad30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3004006fd0 tx=0x7f3004006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:09.846+0000 7f2fff7fe700 1 -- 172.21.15.27:0/2314920128 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3000053060 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.071+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f301010d0f0 con 0x7f301007ce50 2023-12-17T12:02:10.709 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.075+0000 7f2fff7fe700 1 -- 172.21.15.27:0/2314920128 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f301010d0f0 con 0x7f301007ce50 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.078+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f2ff8038870 msgr2=0x7f2ff803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.078+0000 7f30149b0700 1 --2- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f2ff8038870 0x7f2ff803ad30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3004006fd0 tx=0x7f3004006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.078+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 msgr2=0x7f301019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.078+0000 7f30149b0700 1 --2- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301019caa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3000003f70 tx=0x7f30000119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.079+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 shutdown_connections 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.079+0000 7f30149b0700 1 --2- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f2ff8038870 0x7f2ff803ad30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.079+0000 7f30149b0700 1 --2- 172.21.15.27:0/2314920128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301007ce50 0x7f301019caa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.079+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 >> 172.21.15.27:0/2314920128 conn(0x7f3010102140 msgr2=0x7f3010102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.079+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 shutdown_connections 2023-12-17T12:02:10.710 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:10.079+0000 7f30149b0700 1 -- 172.21.15.27:0/2314920128 wait complete. 2023-12-17T12:02:11.260 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:11 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2314920128' entity='client.admin' 2023-12-17T12:02:11.260 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:11 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:02:11.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.287+0000 7f457d2d9700 1 Processor -- start 2023-12-17T12:02:11.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.287+0000 7f457d2d9700 1 -- start start 2023-12-17T12:02:11.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.288+0000 7f457d2d9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457807b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:11.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.288+0000 7f457d2d9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457807b880 con 0x7f457807ce50 2023-12-17T12:02:11.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.288+0000 7f4577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457807b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:11.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.288+0000 7f4577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457807b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43694/0 (socket says 172.21.15.27:43694) 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.288+0000 7f4577fff700 1 -- 172.21.15.27:0/1985539678 learned_addr learned my addr 172.21.15.27:0/1985539678 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.289+0000 7f4577fff700 1 -- 172.21.15.27:0/1985539678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457807b9c0 con 0x7f457807ce50 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.289+0000 7f4577fff700 1 --2- 172.21.15.27:0/1985539678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457807b2b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45680098a0 tx=0x7f4568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cb529ba00f3566 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.289+0000 7f4576ffd700 1 -- 172.21.15.27:0/1985539678 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4568004160 con 0x7f457807ce50 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.289+0000 7f4576ffd700 1 -- 172.21.15.27:0/1985539678 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45680042c0 con 0x7f457807ce50 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.289+0000 7f4576ffd700 1 -- 172.21.15.27:0/1985539678 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45680118e0 con 0x7f457807ce50 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.290+0000 7f457d2d9700 1 -- 172.21.15.27:0/1985539678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 msgr2=0x7f457807b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.290+0000 7f457d2d9700 1 --2- 172.21.15.27:0/1985539678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457807b2b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45680098a0 tx=0x7f4568009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.290+0000 7f457d2d9700 1 -- 172.21.15.27:0/1985539678 shutdown_connections 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.290+0000 7f457d2d9700 1 --2- 172.21.15.27:0/1985539678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457807b2b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.290+0000 7f457d2d9700 1 -- 172.21.15.27:0/1985539678 >> 172.21.15.27:0/1985539678 conn(0x7f4578101f90 msgr2=0x7f45781043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.290+0000 7f457d2d9700 1 -- 172.21.15.27:0/1985539678 shutdown_connections 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.291+0000 7f457d2d9700 1 -- 172.21.15.27:0/1985539678 wait complete. 2023-12-17T12:02:11.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.292+0000 7f457d2d9700 1 Processor -- start 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.292+0000 7f457d2d9700 1 -- start start 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.292+0000 7f457d2d9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.292+0000 7f457d2d9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457819ce10 con 0x7f457807ce50 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.292+0000 7f4577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.293+0000 7f4577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43710/0 (socket says 172.21.15.27:43710) 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.293+0000 7f4577fff700 1 -- 172.21.15.27:0/795872921 learned_addr learned my addr 172.21.15.27:0/795872921 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.293+0000 7f4577fff700 1 -- 172.21.15.27:0/795872921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568009550 con 0x7f457807ce50 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.293+0000 7f4577fff700 1 --2- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457819c8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4568003f70 tx=0x7f45680119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.293+0000 7f45757fa700 1 -- 172.21.15.27:0/795872921 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45680037d0 con 0x7f457807ce50 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.294+0000 7f45757fa700 1 -- 172.21.15.27:0/795872921 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f456801b440 con 0x7f457807ce50 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.294+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457819d010 con 0x7f457807ce50 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.294+0000 7f45757fa700 1 -- 172.21.15.27:0/795872921 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45680193d0 con 0x7f457807ce50 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.294+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457819d4b0 con 0x7f457807ce50 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.295+0000 7f45757fa700 1 -- 172.21.15.27:0/795872921 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f456801b860 con 0x7f457807ce50 2023-12-17T12:02:11.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.295+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457810cd30 con 0x7f457807ce50 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.296+0000 7f45757fa700 1 --2- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f4560038870 0x7f456003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.296+0000 7f45757fa700 1 -- 172.21.15.27:0/795872921 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f456804c6d0 con 0x7f457807ce50 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.296+0000 7f45777fe700 1 --2- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f4560038870 0x7f456003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.297+0000 7f45777fe700 1 --2- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f4560038870 0x7f456003ad30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f456c006fd0 tx=0x7f456c006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.301+0000 7f45757fa700 1 -- 172.21.15.27:0/795872921 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4568018bd0 con 0x7f457807ce50 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.514+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f457819fda0 con 0x7f457807ce50 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.518+0000 7f45757fa700 1 -- 172.21.15.27:0/795872921 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f456801f0c0 con 0x7f457807ce50 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f4560038870 msgr2=0x7f456003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 --2- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f4560038870 0x7f456003ad30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f456c006fd0 tx=0x7f456c006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 msgr2=0x7f457819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 --2- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457819c8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4568003f70 tx=0x7f45680119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 shutdown_connections 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 --2- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f4560038870 0x7f456003ad30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:11.837 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 --2- 172.21.15.27:0/795872921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f457807ce50 0x7f457819c8d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:11.838 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 >> 172.21.15.27:0/795872921 conn(0x7f4578101f90 msgr2=0x7f4578102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:11.838 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 shutdown_connections 2023-12-17T12:02:11.838 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:11.521+0000 7f457d2d9700 1 -- 172.21.15.27:0/795872921 wait complete. 2023-12-17T12:02:11.838 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling the dashboard module... 2023-12-17T12:02:12.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:12 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/795872921' entity='client.admin' 2023-12-17T12:02:12.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:12 smithi027 ceph-mon[36871]: from='mgr.14122 172.21.15.27:0/874263209' entity='mgr.x' 2023-12-17T12:02:13.795 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:13 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3503409462' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-17T12:02:13.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: ignoring --setuser ceph since I am not root 2023-12-17T12:02:13.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: ignoring --setgroup ceph since I am not root 2023-12-17T12:02:13.937 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.306+0000 7f53325d3700 1 Processor -- start 2023-12-17T12:02:13.937 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.307+0000 7f53325d3700 1 -- start start 2023-12-17T12:02:13.937 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.307+0000 7f53325d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c07b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:13.937 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.307+0000 7f53325d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c07b860 con 0x7f532c07ce30 2023-12-17T12:02:13.937 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.307+0000 7f532bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c07b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:13.937 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.307+0000 7f532bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c07b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43712/0 (socket says 172.21.15.27:43712) 2023-12-17T12:02:13.937 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.307+0000 7f532bfff700 1 -- 172.21.15.27:0/667061510 learned_addr learned my addr 172.21.15.27:0/667061510 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:13.937 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.308+0000 7f532bfff700 1 -- 172.21.15.27:0/667061510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c07b9a0 con 0x7f532c07ce30 2023-12-17T12:02:13.938 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.308+0000 7f532bfff700 1 --2- 172.21.15.27:0/667061510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c07b290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f531c0098a0 tx=0x7f531c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc9b84be9a1a002 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:13.938 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.308+0000 7f532affd700 1 -- 172.21.15.27:0/667061510 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f531c004160 con 0x7f532c07ce30 2023-12-17T12:02:13.938 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.308+0000 7f532affd700 1 -- 172.21.15.27:0/667061510 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f531c0042c0 con 0x7f532c07ce30 2023-12-17T12:02:13.938 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.308+0000 7f532affd700 1 -- 172.21.15.27:0/667061510 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f531c0118e0 con 0x7f532c07ce30 2023-12-17T12:02:13.938 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.309+0000 7f53325d3700 1 -- 172.21.15.27:0/667061510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 msgr2=0x7f532c07b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:13.938 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.309+0000 7f53325d3700 1 --2- 172.21.15.27:0/667061510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c07b290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f531c0098a0 tx=0x7f531c009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:13.938 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.309+0000 7f53325d3700 1 -- 172.21.15.27:0/667061510 shutdown_connections 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.309+0000 7f53325d3700 1 --2- 172.21.15.27:0/667061510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c07b290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.309+0000 7f53325d3700 1 -- 172.21.15.27:0/667061510 >> 172.21.15.27:0/667061510 conn(0x7f532c102170 msgr2=0x7f532c104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.309+0000 7f53325d3700 1 -- 172.21.15.27:0/667061510 shutdown_connections 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.309+0000 7f53325d3700 1 -- 172.21.15.27:0/667061510 wait complete. 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.311+0000 7f53325d3700 1 Processor -- start 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.311+0000 7f53325d3700 1 -- start start 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.311+0000 7f53325d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.311+0000 7f53325d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c19cf70 con 0x7f532c07ce30 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.311+0000 7f532bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.311+0000 7f532bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43722/0 (socket says 172.21.15.27:43722) 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.311+0000 7f532bfff700 1 -- 172.21.15.27:0/3503409462 learned_addr learned my addr 172.21.15.27:0/3503409462 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:13.939 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.312+0000 7f532bfff700 1 -- 172.21.15.27:0/3503409462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c009550 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.312+0000 7f532bfff700 1 --2- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c19ca30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f531c003f70 tx=0x7f531c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.312+0000 7f53297fa700 1 -- 172.21.15.27:0/3503409462 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f531c0036a0 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.313+0000 7f53297fa700 1 -- 172.21.15.27:0/3503409462 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f531c01b440 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.313+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c19d170 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.313+0000 7f53297fa700 1 -- 172.21.15.27:0/3503409462 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f531c023400 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.313+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c19d610 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.314+0000 7f53297fa700 1 -- 172.21.15.27:0/3503409462 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f531c023630 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.314+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5318005320 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.315+0000 7f53297fa700 1 --2- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5314038870 0x7f531403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.315+0000 7f53297fa700 1 -- 172.21.15.27:0/3503409462 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f531c04c760 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.315+0000 7f532b7fe700 1 --2- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5314038870 0x7f531403ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.316+0000 7f532b7fe700 1 --2- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5314038870 0x7f531403ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5320006fd0 tx=0x7f5320006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.320+0000 7f53297fa700 1 -- 172.21.15.27:0/3503409462 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f531c018350 con 0x7f532c07ce30 2023-12-17T12:02:13.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:12.609+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f5318005190 con 0x7f532c07ce30 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.525+0000 7f53297fa700 1 -- 172.21.15.27:0/3503409462 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f531c02bb40 con 0x7f532c07ce30 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.527+0000 7f53297fa700 1 -- 172.21.15.27:0/3503409462 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f531c01b5b0 con 0x7f532c07ce30 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.529+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5314038870 msgr2=0x7f531403ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.529+0000 7f53325d3700 1 --2- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5314038870 0x7f531403ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5320006fd0 tx=0x7f5320006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.529+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 msgr2=0x7f532c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.529+0000 7f53325d3700 1 --2- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c19ca30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f531c003f70 tx=0x7f531c0119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.530+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 shutdown_connections 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.530+0000 7f53325d3700 1 --2- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5314038870 0x7f531403ad30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.530+0000 7f53325d3700 1 --2- 172.21.15.27:0/3503409462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f532c07ce30 0x7f532c19ca30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.530+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 >> 172.21.15.27:0/3503409462 conn(0x7f532c102170 msgr2=0x7f532c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.531+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 shutdown_connections 2023-12-17T12:02:13.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:13.531+0000 7f53325d3700 1 -- 172.21.15.27:0/3503409462 wait complete. 2023-12-17T12:02:14.379 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:14.247+0000 7fcf1e3a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-17T12:02:14.379 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:14.377+0000 7fcf1e3a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-17T12:02:14.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:14 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3503409462' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-17T12:02:14.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:14 smithi027 ceph-mon[36871]: mgrmap e9: x(active, since 18s) 2023-12-17T12:02:14.663 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:14.506+0000 7fcf1e3a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-17T12:02:14.663 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:14.625+0000 7fcf1e3a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-17T12:02:14.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:02:14.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-17T12:02:14.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-17T12:02:14.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-17T12:02:14.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-17T12:02:14.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:02:14.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.391+0000 7f5c202fb700 1 Processor -- start 2023-12-17T12:02:14.994 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.391+0000 7f5c202fb700 1 -- start start 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.391+0000 7f5c202fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1807b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.391+0000 7f5c202fb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1807b880 con 0x7f5c1807ce50 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.392+0000 7f5c1e097700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1807b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.392+0000 7f5c1e097700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1807b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43750/0 (socket says 172.21.15.27:43750) 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.392+0000 7f5c1e097700 1 -- 172.21.15.27:0/1945875647 learned_addr learned my addr 172.21.15.27:0/1945875647 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.392+0000 7f5c1e097700 1 -- 172.21.15.27:0/1945875647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1807b9c0 con 0x7f5c1807ce50 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.392+0000 7f5c1e097700 1 --2- 172.21.15.27:0/1945875647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1807b2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c140098a0 tx=0x7f5c14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc791ac173b58ef3 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.393+0000 7f5c1d095700 1 -- 172.21.15.27:0/1945875647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c14004160 con 0x7f5c1807ce50 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.393+0000 7f5c1d095700 1 -- 172.21.15.27:0/1945875647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c140042c0 con 0x7f5c1807ce50 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.393+0000 7f5c1d095700 1 -- 172.21.15.27:0/1945875647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c140118e0 con 0x7f5c1807ce50 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.393+0000 7f5c202fb700 1 -- 172.21.15.27:0/1945875647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 msgr2=0x7f5c1807b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.393+0000 7f5c202fb700 1 --2- 172.21.15.27:0/1945875647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1807b2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c140098a0 tx=0x7f5c14009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.394+0000 7f5c202fb700 1 -- 172.21.15.27:0/1945875647 shutdown_connections 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.394+0000 7f5c202fb700 1 --2- 172.21.15.27:0/1945875647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1807b2b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:14.995 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.394+0000 7f5c202fb700 1 -- 172.21.15.27:0/1945875647 >> 172.21.15.27:0/1945875647 conn(0x7f5c18102190 msgr2=0x7f5c181045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.394+0000 7f5c202fb700 1 -- 172.21.15.27:0/1945875647 shutdown_connections 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.394+0000 7f5c202fb700 1 -- 172.21.15.27:0/1945875647 wait complete. 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.395+0000 7f5c202fb700 1 Processor -- start 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.395+0000 7f5c202fb700 1 -- start start 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.395+0000 7f5c202fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.395+0000 7f5c202fb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1819cf80 con 0x7f5c1807ce50 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.396+0000 7f5c1e097700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.396+0000 7f5c1e097700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43760/0 (socket says 172.21.15.27:43760) 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.396+0000 7f5c1e097700 1 -- 172.21.15.27:0/1340951564 learned_addr learned my addr 172.21.15.27:0/1340951564 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.396+0000 7f5c1e097700 1 -- 172.21.15.27:0/1340951564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14009550 con 0x7f5c1807ce50 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.396+0000 7f5c1e097700 1 --2- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1819ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c14003f70 tx=0x7f5c140119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.397+0000 7f5c0f7fe700 1 -- 172.21.15.27:0/1340951564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c140036a0 con 0x7f5c1807ce50 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.397+0000 7f5c0f7fe700 1 -- 172.21.15.27:0/1340951564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c1401b440 con 0x7f5c1807ce50 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.397+0000 7f5c0f7fe700 1 -- 172.21.15.27:0/1340951564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c14023400 con 0x7f5c1807ce50 2023-12-17T12:02:14.996 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.397+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1819d180 con 0x7f5c1807ce50 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.397+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1819d620 con 0x7f5c1807ce50 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.398+0000 7f5c0f7fe700 1 -- 172.21.15.27:0/1340951564 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f5c1401b5b0 con 0x7f5c1807ce50 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.398+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1810cf30 con 0x7f5c1807ce50 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.399+0000 7f5c0f7fe700 1 --2- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5c040388c0 0x7f5c0403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.399+0000 7f5c0f7fe700 1 -- 172.21.15.27:0/1340951564 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5c1404c840 con 0x7f5c1807ce50 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.399+0000 7f5c1d896700 1 -- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5c040388c0 msgr2=0x7f5c0403ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/674094332 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.399+0000 7f5c1d896700 1 --2- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5c040388c0 0x7f5c0403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.408+0000 7f5c0f7fe700 1 -- 172.21.15.27:0/1340951564 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5c1400be40 con 0x7f5c1807ce50 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.600+0000 7f5c1d896700 1 -- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5c040388c0 msgr2=0x7f5c0403ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/674094332 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.600+0000 7f5c1d896700 1 --2- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5c040388c0 0x7f5c0403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.681+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f5c1819feb0 con 0x7f5c1807ce50 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.681+0000 7f5c0f7fe700 1 -- 172.21.15.27:0/1340951564 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5c1400b3e0 con 0x7f5c1807ce50 2023-12-17T12:02:14.997 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5c040388c0 msgr2=0x7f5c0403ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 --2- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5c040388c0 0x7f5c0403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 msgr2=0x7f5c1819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 --2- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1819ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c14003f70 tx=0x7f5c140119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 shutdown_connections 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 --2- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f5c040388c0 0x7f5c0403ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 --2- 172.21.15.27:0/1340951564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1807ce50 0x7f5c1819ca40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 >> 172.21.15.27:0/1340951564 conn(0x7f5c18102190 msgr2=0x7f5c18102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 shutdown_connections 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:14.684+0000 7f5c202fb700 1 -- 172.21.15.27:0/1340951564 wait complete. 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2023-12-17T12:02:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 9... 2023-12-17T12:02:15.860 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:15 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1340951564' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-17T12:02:15.860 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:15 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:15.538+0000 7fcf1e3a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:02:16.125 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:15 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:15.858+0000 7fcf1e3a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-17T12:02:16.125 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:15 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:15.976+0000 7fcf1e3a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-17T12:02:16.125 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:16.123+0000 7fcf1e3a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-17T12:02:16.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:16.328+0000 7fcf1e3a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-17T12:02:16.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:16.507+0000 7fcf1e3a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-17T12:02:16.913 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:16.626+0000 7fcf1e3a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-17T12:02:17.413 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:17.123+0000 7fcf1e3a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-17T12:02:17.869 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:17.493+0000 7fcf1e3a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-17T12:02:18.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:17.867+0000 7fcf1e3a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-17T12:02:18.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:18 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:17.999+0000 7fcf1e3a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-17T12:02:19.346 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:19 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:19.006+0000 7fcf1e3a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-17T12:02:19.347 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:19 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:19.138+0000 7fcf1e3a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-17T12:02:19.347 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:19 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:19.344+0000 7fcf1e3a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-17T12:02:20.467 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:20.212+0000 7fcf1e3a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-17T12:02:21.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:20.464+0000 7fcf1e3a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-17T12:02:21.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:20.736+0000 7fcf1e3a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-17T12:02:22.320 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:21 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:21.939+0000 7fcf1e3a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-17T12:02:22.663 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:22.317+0000 7fcf1e3a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:02:22.663 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:22.444+0000 7fcf1e3a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-17T12:02:23.163 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:02:22.713+0000 7fcf1e3a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-17T12:02:23.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: Active manager daemon x restarted 2023-12-17T12:02:23.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: Activating manager daemon x 2023-12-17T12:02:23.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: osdmap e3: 0 total, 0 up, 0 in 2023-12-17T12:02:23.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: mgrmap e10: x(active, starting, since 0.00424954s) 2023-12-17T12:02:23.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:02:23.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-17T12:02:23.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-17T12:02:23.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-17T12:02:23.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-17T12:02:23.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: Manager daemon x is now available 2023-12-17T12:02:23.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:02:24.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:02:24.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-17T12:02:24.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-17T12:02:24.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:23 smithi027 ceph-mon[36871]: [17/Dec/2023:12:02:23] ENGINE Bus STARTING 2023-12-17T12:02:24.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:23 smithi027 ceph-mon[36871]: [17/Dec/2023:12:02:23] ENGINE Serving on https://172.21.15.27:7150 2023-12-17T12:02:24.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:23 smithi027 ceph-mon[36871]: mgrmap e11: x(active, since 1.008s) 2023-12-17T12:02:24.297 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2023-12-17T12:02:24.297 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-17T12:02:24.297 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.450+0000 7f0559142700 1 Processor -- start 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.451+0000 7f0559142700 1 -- start start 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.451+0000 7f0559142700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f0554109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.451+0000 7f0559142700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055407ad80 con 0x7f05541076b0 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.452+0000 7f0552d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f0554109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.452+0000 7f0552d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f0554109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43776/0 (socket says 172.21.15.27:43776) 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.452+0000 7f0552d9d700 1 -- 172.21.15.27:0/3925288985 learned_addr learned my addr 172.21.15.27:0/3925288985 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.452+0000 7f0552d9d700 1 -- 172.21.15.27:0/3925288985 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055407aec0 con 0x7f05541076b0 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.452+0000 7f0552d9d700 1 --2- 172.21.15.27:0/3925288985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f0554109aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f05440098a0 tx=0x7f0544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268e599afd00e2a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.453+0000 7f0551d9b700 1 -- 172.21.15.27:0/3925288985 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0544004160 con 0x7f05541076b0 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.453+0000 7f0551d9b700 1 -- 172.21.15.27:0/3925288985 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05440042c0 con 0x7f05541076b0 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.453+0000 7f0551d9b700 1 -- 172.21.15.27:0/3925288985 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05440118e0 con 0x7f05541076b0 2023-12-17T12:02:24.298 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.453+0000 7f0559142700 1 -- 172.21.15.27:0/3925288985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 msgr2=0x7f0554109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.453+0000 7f0559142700 1 --2- 172.21.15.27:0/3925288985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f0554109aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f05440098a0 tx=0x7f0544009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.454+0000 7f0559142700 1 -- 172.21.15.27:0/3925288985 shutdown_connections 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.454+0000 7f0559142700 1 --2- 172.21.15.27:0/3925288985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f0554109aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.454+0000 7f0559142700 1 -- 172.21.15.27:0/3925288985 >> 172.21.15.27:0/3925288985 conn(0x7f0554101020 msgr2=0x7f0554103440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.454+0000 7f0559142700 1 -- 172.21.15.27:0/3925288985 shutdown_connections 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.454+0000 7f0559142700 1 -- 172.21.15.27:0/3925288985 wait complete. 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.455+0000 7f0559142700 1 Processor -- start 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.455+0000 7f0559142700 1 -- start start 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.455+0000 7f0559142700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f055419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.455+0000 7f0559142700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055419d000 con 0x7f05541076b0 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.455+0000 7f0552d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f055419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.456+0000 7f0552d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f055419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43778/0 (socket says 172.21.15.27:43778) 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.456+0000 7f0552d9d700 1 -- 172.21.15.27:0/4145293482 learned_addr learned my addr 172.21.15.27:0/4145293482 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.456+0000 7f0552d9d700 1 -- 172.21.15.27:0/4145293482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0544009550 con 0x7f05541076b0 2023-12-17T12:02:24.299 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.456+0000 7f0552d9d700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f055419cac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f05440046e0 tx=0x7f05440119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.457+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05440037d0 con 0x7f05541076b0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.457+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f054401b440 con 0x7f05541076b0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.457+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055419d200 con 0x7f05541076b0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.457+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0544023400 con 0x7f05541076b0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.457+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055419d6a0 con 0x7f05541076b0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.458+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f0544003930 con 0x7f05541076b0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.459+0000 7f053bfff700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.459+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 --> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f053c03b490 con 0x7f053c0388c0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.459+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f054404c890 con 0x7f05541076b0 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.459+0000 7f055259c700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 msgr2=0x7f053c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/674094332 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.459+0000 7f055259c700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.660+0000 7f055259c700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 msgr2=0x7f053c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/674094332 2023-12-17T12:02:24.300 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:15.660+0000 7f055259c700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:16.061+0000 7f055259c700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 msgr2=0x7f053c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/674094332 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:16.061+0000 7f055259c700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:16.862+0000 7f055259c700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 msgr2=0x7f053c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/674094332 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:16.862+0000 7f055259c700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:18.463+0000 7f055259c700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 msgr2=0x7f053c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/674094332 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:18.463+0000 7f055259c700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:21.666+0000 7f055259c700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 msgr2=0x7f053c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/674094332 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:21.666+0000 7f055259c700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:22.718+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 10) v1 ==== 45360+0+0 (secure 0 0 0) 0x7f054401b840 con 0x7f05541076b0 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:22.718+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 msgr2=0x7f053c03ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:22.718+0000 7f053bfff700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.720+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 11) v1 ==== 45487+0+0 (secure 0 0 0) 0x7f0544020d70 con 0x7f05541076b0 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.721+0000 7f053bfff700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f053c03c360 0x7f053c03e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:24.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.721+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f053c03b490 con 0x7f053c03c360 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.722+0000 7f055259c700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f053c03c360 0x7f053c03e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.723+0000 7f055259c700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f053c03c360 0x7f053c03e750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0548003a10 tx=0x7f0548009270 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.724+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f053c03b490 con 0x7f053c03c360 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.732+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f055410acf0 con 0x7f053c03c360 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f053bfff700 1 -- 172.21.15.27:0/4145293482 <== mgr.14154 v2:172.21.15.27:6800/46170456 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f055410acf0 con 0x7f053c03c360 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f053c03c360 msgr2=0x7f053c03e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f053c03c360 0x7f053c03e750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0548003a10 tx=0x7f0548009270 comp rx=0 tx=0).stop 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 msgr2=0x7f055419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f055419cac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f05440046e0 tx=0x7f05440119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 shutdown_connections 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f053c03c360 0x7f053c03e750 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05541076b0 0x7f055419cac0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 --2- 172.21.15.27:0/4145293482 >> [v2:172.21.15.27:6800/674094332,v1:172.21.15.27:6801/674094332] conn(0x7f053c0388c0 0x7f053c03ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:24.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.733+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 >> 172.21.15.27:0/4145293482 conn(0x7f0554101020 msgr2=0x7f0554101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:24.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.734+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 shutdown_connections 2023-12-17T12:02:24.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:23.734+0000 7f0559142700 1 -- 172.21.15.27:0/4145293482 wait complete. 2023-12-17T12:02:24.303 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 9 is available 2023-12-17T12:02:24.303 INFO:teuthology.orchestra.run.smithi027.stdout:Generating a dashboard self-signed certificate... 2023-12-17T12:02:25.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:25 smithi027 ceph-mon[36871]: [17/Dec/2023:12:02:23] ENGINE Serving on http://172.21.15.27:8765 2023-12-17T12:02:25.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:25 smithi027 ceph-mon[36871]: [17/Dec/2023:12:02:23] ENGINE Bus STARTED 2023-12-17T12:02:25.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:25 smithi027 ceph-mon[36871]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-17T12:02:25.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:25 smithi027 ceph-mon[36871]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-17T12:02:25.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:25.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:25.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:25.437 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-17T12:02:25.437 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.746+0000 7f9bfc65e700 1 Processor -- start 2023-12-17T12:02:25.437 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.746+0000 7f9bfc65e700 1 -- start start 2023-12-17T12:02:25.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.746+0000 7f9bfc65e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:25.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.746+0000 7f9bfc65e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4107530 con 0x7f9bf4106b40 2023-12-17T12:02:25.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.747+0000 7f9bfa3fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:25.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.747+0000 7f9bfa3fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf4106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58394/0 (socket says 172.21.15.27:58394) 2023-12-17T12:02:25.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.747+0000 7f9bfa3fa700 1 -- 172.21.15.27:0/1716360152 learned_addr learned my addr 172.21.15.27:0/1716360152 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:25.438 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.747+0000 7f9bfa3fa700 1 -- 172.21.15.27:0/1716360152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4107d40 con 0x7f9bf4106b40 2023-12-17T12:02:25.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.748+0000 7f9bfa3fa700 1 --2- 172.21.15.27:0/1716360152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf4106f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00098a0 tx=0x7f9bf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92be2361b746473 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:25.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.748+0000 7f9bf93f8700 1 -- 172.21.15.27:0/1716360152 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bf0004160 con 0x7f9bf4106b40 2023-12-17T12:02:25.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.748+0000 7f9bf93f8700 1 -- 172.21.15.27:0/1716360152 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bf00042c0 con 0x7f9bf4106b40 2023-12-17T12:02:25.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.748+0000 7f9bf93f8700 1 -- 172.21.15.27:0/1716360152 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bf00118e0 con 0x7f9bf4106b40 2023-12-17T12:02:25.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.749+0000 7f9bfc65e700 1 -- 172.21.15.27:0/1716360152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 msgr2=0x7f9bf4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:25.439 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.749+0000 7f9bfc65e700 1 --2- 172.21.15.27:0/1716360152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf4106f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00098a0 tx=0x7f9bf0009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:25.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.749+0000 7f9bfc65e700 1 -- 172.21.15.27:0/1716360152 shutdown_connections 2023-12-17T12:02:25.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.749+0000 7f9bfc65e700 1 --2- 172.21.15.27:0/1716360152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf4106f60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:25.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.749+0000 7f9bfc65e700 1 -- 172.21.15.27:0/1716360152 >> 172.21.15.27:0/1716360152 conn(0x7f9bf4102140 msgr2=0x7f9bf4104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:25.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.749+0000 7f9bfc65e700 1 -- 172.21.15.27:0/1716360152 shutdown_connections 2023-12-17T12:02:25.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.749+0000 7f9bfc65e700 1 -- 172.21.15.27:0/1716360152 wait complete. 2023-12-17T12:02:25.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.751+0000 7f9bfc65e700 1 Processor -- start 2023-12-17T12:02:25.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.751+0000 7f9bfc65e700 1 -- start start 2023-12-17T12:02:25.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.751+0000 7f9bfc65e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:25.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.751+0000 7f9bfc65e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf419cfb0 con 0x7f9bf4106b40 2023-12-17T12:02:25.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.752+0000 7f9bfa3fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:25.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.752+0000 7f9bfa3fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58402/0 (socket says 172.21.15.27:58402) 2023-12-17T12:02:25.441 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.752+0000 7f9bfa3fa700 1 -- 172.21.15.27:0/2311125538 learned_addr learned my addr 172.21.15.27:0/2311125538 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:25.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.752+0000 7f9bfa3fa700 1 -- 172.21.15.27:0/2311125538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009550 con 0x7f9bf4106b40 2023-12-17T12:02:25.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.752+0000 7f9bfa3fa700 1 --2- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf419ca70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0003f70 tx=0x7f9bf00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:25.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.753+0000 7f9beb7fe700 1 -- 172.21.15.27:0/2311125538 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bf00036a0 con 0x7f9bf4106b40 2023-12-17T12:02:25.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.753+0000 7f9beb7fe700 1 -- 172.21.15.27:0/2311125538 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bf001b440 con 0x7f9bf4106b40 2023-12-17T12:02:25.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.753+0000 7f9beb7fe700 1 -- 172.21.15.27:0/2311125538 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bf0023400 con 0x7f9bf4106b40 2023-12-17T12:02:25.442 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.753+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf419d1b0 con 0x7f9bf4106b40 2023-12-17T12:02:25.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.753+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf419d650 con 0x7f9bf4106b40 2023-12-17T12:02:25.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.754+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf410af70 con 0x7f9bf4106b40 2023-12-17T12:02:25.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.755+0000 7f9beb7fe700 1 -- 172.21.15.27:0/2311125538 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) v1 ==== 45487+0+0 (secure 0 0 0) 0x7f9bf001b5b0 con 0x7f9bf4106b40 2023-12-17T12:02:25.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.755+0000 7f9beb7fe700 1 --2- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9be0038790 0x7f9be003ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:25.443 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.755+0000 7f9beb7fe700 1 -- 172.21.15.27:0/2311125538 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9bf004c690 con 0x7f9bf4106b40 2023-12-17T12:02:25.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.756+0000 7f9bf9bf9700 1 --2- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9be0038790 0x7f9be003ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:25.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.756+0000 7f9bf9bf9700 1 --2- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9be0038790 0x7f9be003ac50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9be4006fd0 tx=0x7f9be4006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:25.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.765+0000 7f9beb7fe700 1 -- 172.21.15.27:0/2311125538 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9bf000be40 con 0x7f9bf4106b40 2023-12-17T12:02:25.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:24.983+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bf419ff40 con 0x7f9be0038790 2023-12-17T12:02:25.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.082+0000 7f9beb7fe700 1 -- 172.21.15.27:0/2311125538 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f9bf419ff40 con 0x7f9be0038790 2023-12-17T12:02:25.444 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9be0038790 msgr2=0x7f9be003ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:25.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 --2- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9be0038790 0x7f9be003ac50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9be4006fd0 tx=0x7f9be4006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:25.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 msgr2=0x7f9bf419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:25.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 --2- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf419ca70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0003f70 tx=0x7f9bf00119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:25.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 shutdown_connections 2023-12-17T12:02:25.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 --2- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9be0038790 0x7f9be003ac50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:25.445 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 --2- 172.21.15.27:0/2311125538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bf4106b40 0x7f9bf419ca70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:25.446 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 >> 172.21.15.27:0/2311125538 conn(0x7f9bf4102140 msgr2=0x7f9bf4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:25.446 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 shutdown_connections 2023-12-17T12:02:25.446 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.085+0000 7f9bfc65e700 1 -- 172.21.15.27:0/2311125538 wait complete. 2023-12-17T12:02:25.446 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial admin user... 2023-12-17T12:02:26.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:26 smithi027 ceph-mon[36871]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:26.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:26 smithi027 ceph-mon[36871]: mgrmap e12: x(active, since 2s) 2023-12-17T12:02:26.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:27.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jNMVBIKEohSlgnIZLt3dSOsgxkHIPgDYb1d80K1gW6STKQFge20M.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1702814546, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.921+0000 7f6946382700 1 Processor -- start 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.921+0000 7f6946382700 1 -- start start 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.921+0000 7f6946382700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694007b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.921+0000 7f6946382700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694007b860 con 0x7f694007ce30 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.922+0000 7f693ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694007b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.922+0000 7f693ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694007b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58412/0 (socket says 172.21.15.27:58412) 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.922+0000 7f693ffff700 1 -- 172.21.15.27:0/1716855202 learned_addr learned my addr 172.21.15.27:0/1716855202 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.922+0000 7f693ffff700 1 -- 172.21.15.27:0/1716855202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694007b9a0 con 0x7f694007ce30 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.923+0000 7f693ffff700 1 --2- 172.21.15.27:0/1716855202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694007b290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69300098a0 tx=0x7f6930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2c85386d0aca62 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.923+0000 7f693effd700 1 -- 172.21.15.27:0/1716855202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6930004160 con 0x7f694007ce30 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.923+0000 7f693effd700 1 -- 172.21.15.27:0/1716855202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69300042c0 con 0x7f694007ce30 2023-12-17T12:02:27.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.923+0000 7f693effd700 1 -- 172.21.15.27:0/1716855202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69300118e0 con 0x7f694007ce30 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.924+0000 7f6946382700 1 -- 172.21.15.27:0/1716855202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 msgr2=0x7f694007b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.924+0000 7f6946382700 1 --2- 172.21.15.27:0/1716855202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694007b290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69300098a0 tx=0x7f6930009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.924+0000 7f6946382700 1 -- 172.21.15.27:0/1716855202 shutdown_connections 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.924+0000 7f6946382700 1 --2- 172.21.15.27:0/1716855202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694007b290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.924+0000 7f6946382700 1 -- 172.21.15.27:0/1716855202 >> 172.21.15.27:0/1716855202 conn(0x7f6940102190 msgr2=0x7f69401045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.924+0000 7f6946382700 1 -- 172.21.15.27:0/1716855202 shutdown_connections 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.924+0000 7f6946382700 1 -- 172.21.15.27:0/1716855202 wait complete. 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.926+0000 7f6946382700 1 Processor -- start 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.926+0000 7f6946382700 1 -- start start 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.926+0000 7f6946382700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.926+0000 7f6946382700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694019cf70 con 0x7f694007ce30 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.927+0000 7f693ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:27.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.927+0000 7f693ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58420/0 (socket says 172.21.15.27:58420) 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.927+0000 7f693ffff700 1 -- 172.21.15.27:0/1507314408 learned_addr learned my addr 172.21.15.27:0/1507314408 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.927+0000 7f693ffff700 1 -- 172.21.15.27:0/1507314408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6930009550 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.928+0000 7f693ffff700 1 --2- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694019ca30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6930003f70 tx=0x7f69300119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.928+0000 7f693d7fa700 1 -- 172.21.15.27:0/1507314408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69300036a0 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.928+0000 7f693d7fa700 1 -- 172.21.15.27:0/1507314408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f693001b440 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.928+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694019d170 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.928+0000 7f693d7fa700 1 -- 172.21.15.27:0/1507314408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6930023400 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.928+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694019d610 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.929+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694010cf50 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.930+0000 7f693d7fa700 1 -- 172.21.15.27:0/1507314408 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f693001b5b0 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.931+0000 7f693d7fa700 1 --2- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f69280388c0 0x7f692803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.931+0000 7f693d7fa700 1 -- 172.21.15.27:0/1507314408 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f693004c9c0 con 0x7f694007ce30 2023-12-17T12:02:27.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.931+0000 7f693f7fe700 1 --2- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f69280388c0 0x7f692803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.932+0000 7f693f7fe700 1 --2- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f69280388c0 0x7f692803ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6934006fd0 tx=0x7f6934006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:25.940+0000 7f693d7fa700 1 -- 172.21.15.27:0/1507314408 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f693001b8b0 con 0x7f694007ce30 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.184+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6940062a20 con 0x7f69280388c0 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.438+0000 7f693d7fa700 1 -- 172.21.15.27:0/1507314408 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f6940062a20 con 0x7f69280388c0 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f69280388c0 msgr2=0x7f692803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 --2- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f69280388c0 0x7f692803ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6934006fd0 tx=0x7f6934006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 msgr2=0x7f694019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 --2- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694019ca30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6930003f70 tx=0x7f69300119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 shutdown_connections 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 --2- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f69280388c0 0x7f692803ad80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 --2- 172.21.15.27:0/1507314408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f694007ce30 0x7f694019ca30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 >> 172.21.15.27:0/1507314408 conn(0x7f6940102190 msgr2=0x7f6940102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 shutdown_connections 2023-12-17T12:02:27.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:26.441+0000 7f6946382700 1 -- 172.21.15.27:0/1507314408 wait complete. 2023-12-17T12:02:27.225 INFO:teuthology.orchestra.run.smithi027.stdout:Fetching dashboard port number... 2023-12-17T12:02:27.896 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:27 smithi027 ceph-mon[36871]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:27.896 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:27 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:28.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 8443 2023-12-17T12:02:28.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.645+0000 7f1a35715700 1 Processor -- start 2023-12-17T12:02:28.203 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.645+0000 7f1a35715700 1 -- start start 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.645+0000 7f1a35715700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a30106f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.645+0000 7f1a35715700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30107540 con 0x7f1a30106b50 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.646+0000 7f1a2effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a30106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.646+0000 7f1a2effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a30106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54316/0 (socket says 172.21.15.27:54316) 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.646+0000 7f1a2effd700 1 -- 172.21.15.27:0/3195787482 learned_addr learned my addr 172.21.15.27:0/3195787482 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.646+0000 7f1a2effd700 1 -- 172.21.15.27:0/3195787482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a30107d50 con 0x7f1a30106b50 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.646+0000 7f1a2effd700 1 --2- 172.21.15.27:0/3195787482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a30106f70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1a200098a0 tx=0x7f1a20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660887ebdc067d7a server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.646+0000 7f1a2dffb700 1 -- 172.21.15.27:0/3195787482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a20004160 con 0x7f1a30106b50 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a2dffb700 1 -- 172.21.15.27:0/3195787482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a200042c0 con 0x7f1a30106b50 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a2dffb700 1 -- 172.21.15.27:0/3195787482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a200118e0 con 0x7f1a30106b50 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a35715700 1 -- 172.21.15.27:0/3195787482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 msgr2=0x7f1a30106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a35715700 1 --2- 172.21.15.27:0/3195787482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a30106f70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1a200098a0 tx=0x7f1a20009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a35715700 1 -- 172.21.15.27:0/3195787482 shutdown_connections 2023-12-17T12:02:28.204 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a35715700 1 --2- 172.21.15.27:0/3195787482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a30106f70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a35715700 1 -- 172.21.15.27:0/3195787482 >> 172.21.15.27:0/3195787482 conn(0x7f1a30102170 msgr2=0x7f1a30104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a35715700 1 -- 172.21.15.27:0/3195787482 shutdown_connections 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.647+0000 7f1a35715700 1 -- 172.21.15.27:0/3195787482 wait complete. 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.648+0000 7f1a35715700 1 Processor -- start 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.649+0000 7f1a35715700 1 -- start start 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.649+0000 7f1a35715700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a3007cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.649+0000 7f1a35715700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3007d010 con 0x7f1a30106b50 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.649+0000 7f1a2effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a3007cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.649+0000 7f1a2effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a3007cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54332/0 (socket says 172.21.15.27:54332) 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.649+0000 7f1a2effd700 1 -- 172.21.15.27:0/1533790161 learned_addr learned my addr 172.21.15.27:0/1533790161 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.649+0000 7f1a2effd700 1 -- 172.21.15.27:0/1533790161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a20009550 con 0x7f1a30106b50 2023-12-17T12:02:28.205 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.650+0000 7f1a2effd700 1 --2- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a3007cad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a20003f70 tx=0x7f1a200119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.650+0000 7f1a17fff700 1 -- 172.21.15.27:0/1533790161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a200036a0 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.650+0000 7f1a17fff700 1 -- 172.21.15.27:0/1533790161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a2001b440 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.650+0000 7f1a17fff700 1 -- 172.21.15.27:0/1533790161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a20023400 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.650+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3007b250 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.650+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3007b6f0 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.651+0000 7f1a17fff700 1 -- 172.21.15.27:0/1533790161 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f1a20003800 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.651+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a30062c00 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.652+0000 7f1a17fff700 1 --2- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a18038870 0x7f1a1803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.652+0000 7f1a17fff700 1 -- 172.21.15.27:0/1533790161 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1a2004c9e0 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.652+0000 7f1a2e7fc700 1 --2- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a18038870 0x7f1a1803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.653+0000 7f1a2e7fc700 1 --2- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a18038870 0x7f1a1803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a24006fd0 tx=0x7f1a24006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.657+0000 7f1a17fff700 1 -- 172.21.15.27:0/1533790161 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1a2002b350 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.867+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f1a3010af80 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.867+0000 7f1a17fff700 1 -- 172.21.15.27:0/1533790161 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1a2002b350 con 0x7f1a30106b50 2023-12-17T12:02:28.206 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.870+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a18038870 msgr2=0x7f1a1803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.870+0000 7f1a35715700 1 --2- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a18038870 0x7f1a1803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a24006fd0 tx=0x7f1a24006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.870+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 msgr2=0x7f1a3007cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.870+0000 7f1a35715700 1 --2- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a3007cad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a20003f70 tx=0x7f1a200119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.870+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 shutdown_connections 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.870+0000 7f1a35715700 1 --2- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a18038870 0x7f1a1803ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.870+0000 7f1a35715700 1 --2- 172.21.15.27:0/1533790161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a30106b50 0x7f1a3007cad0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.870+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 >> 172.21.15.27:0/1533790161 conn(0x7f1a30102170 msgr2=0x7f1a30102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.871+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 shutdown_connections 2023-12-17T12:02:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:27.871+0000 7f1a35715700 1 -- 172.21.15.27:0/1533790161 wait complete. 2023-12-17T12:02:28.223 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-17T12:02:28.223 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2023-12-17T12:02:28.239 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-17T12:02:28.240 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2023-12-17T12:02:28.240 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2023-12-17T12:02:28.240 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-17T12:02:28.242 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph Dashboard is now available at: 2023-12-17T12:02:28.243 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:28.243 INFO:teuthology.orchestra.run.smithi027.stdout: URL: https://smithi027.front.sepia.ceph.com:8443/ 2023-12-17T12:02:28.243 INFO:teuthology.orchestra.run.smithi027.stdout: User: admin 2023-12-17T12:02:28.243 INFO:teuthology.orchestra.run.smithi027.stdout: Password: wgsi5plexh 2023-12-17T12:02:28.243 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:28.243 INFO:teuthology.orchestra.run.smithi027.stdout:Saving cluster configuration to /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config directory 2023-12-17T12:02:28.244 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling autotune for osd_memory_target 2023-12-17T12:02:28.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:28 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1533790161' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-17T12:02:29.244 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.654+0000 7f8ed013b700 1 Processor -- start 2023-12-17T12:02:29.244 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.654+0000 7f8ed013b700 1 -- start start 2023-12-17T12:02:29.244 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.655+0000 7f8ed013b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec807b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:29.244 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.655+0000 7f8ed013b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec807b840 con 0x7f8ec807ce10 2023-12-17T12:02:29.244 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.655+0000 7f8ecded7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec807b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:29.244 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.655+0000 7f8ecded7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec807b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54338/0 (socket says 172.21.15.27:54338) 2023-12-17T12:02:29.244 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.655+0000 7f8ecded7700 1 -- 172.21.15.27:0/853764775 learned_addr learned my addr 172.21.15.27:0/853764775 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:29.244 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.656+0000 7f8ecded7700 1 -- 172.21.15.27:0/853764775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec807b980 con 0x7f8ec807ce10 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.656+0000 7f8ecded7700 1 --2- 172.21.15.27:0/853764775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec807b270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40098a0 tx=0x7f8ec4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707957d51b071e2f server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.656+0000 7f8ecced5700 1 -- 172.21.15.27:0/853764775 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4004160 con 0x7f8ec807ce10 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.656+0000 7f8ecced5700 1 -- 172.21.15.27:0/853764775 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ec40042c0 con 0x7f8ec807ce10 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.656+0000 7f8ecced5700 1 -- 172.21.15.27:0/853764775 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec40118e0 con 0x7f8ec807ce10 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.657+0000 7f8ed013b700 1 -- 172.21.15.27:0/853764775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 msgr2=0x7f8ec807b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.657+0000 7f8ed013b700 1 --2- 172.21.15.27:0/853764775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec807b270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40098a0 tx=0x7f8ec4009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.657+0000 7f8ed013b700 1 -- 172.21.15.27:0/853764775 shutdown_connections 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.657+0000 7f8ed013b700 1 --2- 172.21.15.27:0/853764775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec807b270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.657+0000 7f8ed013b700 1 -- 172.21.15.27:0/853764775 >> 172.21.15.27:0/853764775 conn(0x7f8ec8102170 msgr2=0x7f8ec8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.657+0000 7f8ed013b700 1 -- 172.21.15.27:0/853764775 shutdown_connections 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.657+0000 7f8ed013b700 1 -- 172.21.15.27:0/853764775 wait complete. 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.658+0000 7f8ed013b700 1 Processor -- start 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.658+0000 7f8ed013b700 1 -- start start 2023-12-17T12:02:29.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.658+0000 7f8ed013b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.658+0000 7f8ed013b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec819cf90 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.659+0000 7f8ecded7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.659+0000 7f8ecded7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54340/0 (socket says 172.21.15.27:54340) 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.659+0000 7f8ecded7700 1 -- 172.21.15.27:0/2822159744 learned_addr learned my addr 172.21.15.27:0/2822159744 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.659+0000 7f8ecded7700 1 -- 172.21.15.27:0/2822159744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec4009550 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.659+0000 7f8ecded7700 1 --2- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec819ca50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4003f70 tx=0x7f8ec40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.660+0000 7f8ebeffd700 1 -- 172.21.15.27:0/2822159744 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec40036a0 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.660+0000 7f8ebeffd700 1 -- 172.21.15.27:0/2822159744 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ec401b440 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.660+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec819d190 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.660+0000 7f8ebeffd700 1 -- 172.21.15.27:0/2822159744 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4023400 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.660+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec819d630 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.661+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec810cef0 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.662+0000 7f8ebeffd700 1 -- 172.21.15.27:0/2822159744 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f8ec401b5b0 con 0x7f8ec807ce10 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.662+0000 7f8ebeffd700 1 --2- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8eb40388c0 0x7f8eb403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:29.246 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.662+0000 7f8ebeffd700 1 -- 172.21.15.27:0/2822159744 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ec404c890 con 0x7f8ec807ce10 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.662+0000 7f8ecd6d6700 1 --2- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8eb40388c0 0x7f8eb403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.663+0000 7f8ecd6d6700 1 --2- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8eb40388c0 0x7f8eb403ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006fd0 tx=0x7f8eb8006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.667+0000 7f8ebeffd700 1 -- 172.21.15.27:0/2822159744 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8ec40195b0 con 0x7f8ec807ce10 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.878+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f8ec819ff20 con 0x7f8ec807ce10 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.878+0000 7f8ebeffd700 1 -- 172.21.15.27:0/2822159744 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8ec40193d0 con 0x7f8ec807ce10 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8eb40388c0 msgr2=0x7f8eb403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 --2- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8eb40388c0 0x7f8eb403ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006fd0 tx=0x7f8eb8006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 msgr2=0x7f8ec819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 --2- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec819ca50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4003f70 tx=0x7f8ec40119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 shutdown_connections 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 --2- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8eb40388c0 0x7f8eb403ad80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 --2- 172.21.15.27:0/2822159744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ec807ce10 0x7f8ec819ca50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 >> 172.21.15.27:0/2822159744 conn(0x7f8ec8102170 msgr2=0x7f8ec8102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 shutdown_connections 2023-12-17T12:02:29.247 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:28.881+0000 7f8ed013b700 1 -- 172.21.15.27:0/2822159744 wait complete. 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.652+0000 7f59e3114700 1 Processor -- start 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.652+0000 7f59e3114700 1 -- start start 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.652+0000 7f59e3114700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc07b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.652+0000 7f59e3114700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc07b860 con 0x7f59dc07ce30 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.653+0000 7f59e0eb0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc07b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.653+0000 7f59e0eb0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc07b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54348/0 (socket says 172.21.15.27:54348) 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.653+0000 7f59e0eb0700 1 -- 172.21.15.27:0/1173042522 learned_addr learned my addr 172.21.15.27:0/1173042522 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.653+0000 7f59e0eb0700 1 -- 172.21.15.27:0/1173042522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc07b9a0 con 0x7f59dc07ce30 2023-12-17T12:02:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.654+0000 7f59e0eb0700 1 --2- 172.21.15.27:0/1173042522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc07b290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59d00098a0 tx=0x7f59d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fb3893a69533c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.654+0000 7f59db7fe700 1 -- 172.21.15.27:0/1173042522 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59d0004160 con 0x7f59dc07ce30 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.654+0000 7f59db7fe700 1 -- 172.21.15.27:0/1173042522 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59d00042c0 con 0x7f59dc07ce30 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.654+0000 7f59db7fe700 1 -- 172.21.15.27:0/1173042522 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59d00118e0 con 0x7f59dc07ce30 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.654+0000 7f59e3114700 1 -- 172.21.15.27:0/1173042522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 msgr2=0x7f59dc07b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.654+0000 7f59e3114700 1 --2- 172.21.15.27:0/1173042522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc07b290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59d00098a0 tx=0x7f59d0009bb0 comp rx=0 tx=0).stop 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.655+0000 7f59e3114700 1 -- 172.21.15.27:0/1173042522 shutdown_connections 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.655+0000 7f59e3114700 1 --2- 172.21.15.27:0/1173042522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc07b290 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.655+0000 7f59e3114700 1 -- 172.21.15.27:0/1173042522 >> 172.21.15.27:0/1173042522 conn(0x7f59dc102170 msgr2=0x7f59dc104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.655+0000 7f59e3114700 1 -- 172.21.15.27:0/1173042522 shutdown_connections 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.655+0000 7f59e3114700 1 -- 172.21.15.27:0/1173042522 wait complete. 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.656+0000 7f59e3114700 1 Processor -- start 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.656+0000 7f59e3114700 1 -- start start 2023-12-17T12:02:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.656+0000 7f59e3114700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.656+0000 7f59e3114700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc19cf70 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.656+0000 7f59e0eb0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.656+0000 7f59e0eb0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54350/0 (socket says 172.21.15.27:54350) 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.656+0000 7f59e0eb0700 1 -- 172.21.15.27:0/13438399 learned_addr learned my addr 172.21.15.27:0/13438399 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.657+0000 7f59e0eb0700 1 -- 172.21.15.27:0/13438399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0009550 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.657+0000 7f59e0eb0700 1 --2- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc19ca30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59d0003f70 tx=0x7f59d00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.657+0000 7f59d9ffb700 1 -- 172.21.15.27:0/13438399 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59d00036a0 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.657+0000 7f59d9ffb700 1 -- 172.21.15.27:0/13438399 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59d001b440 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.657+0000 7f59d9ffb700 1 -- 172.21.15.27:0/13438399 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59d0023400 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.657+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc19d170 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.657+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc19d610 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.658+0000 7f59d9ffb700 1 -- 172.21.15.27:0/13438399 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f59d001b5b0 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.658+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc10cf30 con 0x7f59dc07ce30 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.659+0000 7f59d9ffb700 1 --2- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f59cc038960 0x7f59cc03ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:30.364 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.659+0000 7f59d9ffb700 1 -- 172.21.15.27:0/13438399 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f59d004ca20 con 0x7f59dc07ce30 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.659+0000 7f59dbfff700 1 --2- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f59cc038960 0x7f59cc03ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.660+0000 7f59dbfff700 1 --2- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f59cc038960 0x7f59cc03ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59c8006fd0 tx=0x7f59c8006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.664+0000 7f59d9ffb700 1 -- 172.21.15.27:0/13438399 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f59d0019c30 con 0x7f59dc07ce30 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.989+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f59dc19fea0 con 0x7f59dc07ce30 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.991+0000 7f59d9ffb700 1 -- 172.21.15.27:0/13438399 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f59d0019840 con 0x7f59dc07ce30 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.993+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f59cc038960 msgr2=0x7f59cc03ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.993+0000 7f59e3114700 1 --2- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f59cc038960 0x7f59cc03ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59c8006fd0 tx=0x7f59c8006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.993+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 msgr2=0x7f59dc19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.993+0000 7f59e3114700 1 --2- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc19ca30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59d0003f70 tx=0x7f59d00119a0 comp rx=0 tx=0).stop 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.994+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 shutdown_connections 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.994+0000 7f59e3114700 1 --2- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f59cc038960 0x7f59cc03ae20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.994+0000 7f59e3114700 1 --2- 172.21.15.27:0/13438399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59dc07ce30 0x7f59dc19ca30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.994+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 >> 172.21.15.27:0/13438399 conn(0x7f59dc102170 msgr2=0x7f59dc102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.994+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 shutdown_connections 2023-12-17T12:02:30.365 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2023-12-17T12:02:29.994+0000 7f59e3114700 1 -- 172.21.15.27:0/13438399 wait complete. 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout:Or, if you are only running a single cluster on this host: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: ceph telemetry on 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout:For more information see: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-17T12:02:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:30.367 INFO:teuthology.orchestra.run.smithi027.stdout:Bootstrap complete. 2023-12-17T12:02:30.448 INFO:tasks.cephadm:Fetching config... 2023-12-17T12:02:30.449 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:02:30.449 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-17T12:02:30.536 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-17T12:02:30.536 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:02:30.536 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-17T12:02:30.625 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-17T12:02:30.625 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:02:30.625 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-17T12:02:30.628 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:30 smithi027 ceph-mon[36871]: mgrmap e13: x(active, since 6s) 2023-12-17T12:02:30.629 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:30 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/13438399' entity='client.admin' 2023-12-17T12:02:30.704 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-17T12:02:30.704 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:02:30.704 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-17T12:02:30.789 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-17T12:02:30.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf6OmOd2AJDKBzcKyeXgHHHTikb/7kSocblHmTbIJuY9Vvts6Ri8Tafg7VzwP3W+WGF0v1QBqwZtKZH6EvSjlIqSu3/Fh9yIrgOeRXadfVNh0aiWlBCWNzs4FeedEVJcFKDE4PtMkyZxV6FxeS2pZ8Tzd270FO4JlfxWCwQMMEEfPReZdEdZkygTyGSCDeBGkVpRxl5ovFok4wZsGqRuijO+tYr0Ca92cO4aAZEoaIYYgbneNT+FvBr8t+/IzNoUBiedfnc/OdM0u+uPgVKxzkere9IxkVNxgcUNSeNJEeZDw8/gxGdr4qEl6xgNekERB+1S+E5eV4Rf5mN1An7ZKQRNMOwLuubwVaqijXsh9gtBIGwQyaNFSBQ2maj0B42oywoLxJhDXYgwaDgjoNPz2pmzPYN4sy+RkNTx84WpemtWlSjJ/0pkOqih0za1TM5FAyNv60IWd50WUHJc/lsYyDXo89qTGORqCHf3zyBjw0U0XhxD7ZCLR6u8YLQBNMBE0= ceph-62030760-9cd3-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-17T12:02:30.930 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf6OmOd2AJDKBzcKyeXgHHHTikb/7kSocblHmTbIJuY9Vvts6Ri8Tafg7VzwP3W+WGF0v1QBqwZtKZH6EvSjlIqSu3/Fh9yIrgOeRXadfVNh0aiWlBCWNzs4FeedEVJcFKDE4PtMkyZxV6FxeS2pZ8Tzd270FO4JlfxWCwQMMEEfPReZdEdZkygTyGSCDeBGkVpRxl5ovFok4wZsGqRuijO+tYr0Ca92cO4aAZEoaIYYgbneNT+FvBr8t+/IzNoUBiedfnc/OdM0u+uPgVKxzkere9IxkVNxgcUNSeNJEeZDw8/gxGdr4qEl6xgNekERB+1S+E5eV4Rf5mN1An7ZKQRNMOwLuubwVaqijXsh9gtBIGwQyaNFSBQ2maj0B42oywoLxJhDXYgwaDgjoNPz2pmzPYN4sy+RkNTx84WpemtWlSjJ/0pkOqih0za1TM5FAyNv60IWd50WUHJc/lsYyDXo89qTGORqCHf3zyBjw0U0XhxD7ZCLR6u8YLQBNMBE0= ceph-62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:02:30.965 DEBUG:teuthology.orchestra.run.smithi141:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf6OmOd2AJDKBzcKyeXgHHHTikb/7kSocblHmTbIJuY9Vvts6Ri8Tafg7VzwP3W+WGF0v1QBqwZtKZH6EvSjlIqSu3/Fh9yIrgOeRXadfVNh0aiWlBCWNzs4FeedEVJcFKDE4PtMkyZxV6FxeS2pZ8Tzd270FO4JlfxWCwQMMEEfPReZdEdZkygTyGSCDeBGkVpRxl5ovFok4wZsGqRuijO+tYr0Ca92cO4aAZEoaIYYgbneNT+FvBr8t+/IzNoUBiedfnc/OdM0u+uPgVKxzkere9IxkVNxgcUNSeNJEeZDw8/gxGdr4qEl6xgNekERB+1S+E5eV4Rf5mN1An7ZKQRNMOwLuubwVaqijXsh9gtBIGwQyaNFSBQ2maj0B42oywoLxJhDXYgwaDgjoNPz2pmzPYN4sy+RkNTx84WpemtWlSjJ/0pkOqih0za1TM5FAyNv60IWd50WUHJc/lsYyDXo89qTGORqCHf3zyBjw0U0XhxD7ZCLR6u8YLQBNMBE0= ceph-62030760-9cd3-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-17T12:02:31.087 INFO:teuthology.orchestra.run.smithi141.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf6OmOd2AJDKBzcKyeXgHHHTikb/7kSocblHmTbIJuY9Vvts6Ri8Tafg7VzwP3W+WGF0v1QBqwZtKZH6EvSjlIqSu3/Fh9yIrgOeRXadfVNh0aiWlBCWNzs4FeedEVJcFKDE4PtMkyZxV6FxeS2pZ8Tzd270FO4JlfxWCwQMMEEfPReZdEdZkygTyGSCDeBGkVpRxl5ovFok4wZsGqRuijO+tYr0Ca92cO4aAZEoaIYYgbneNT+FvBr8t+/IzNoUBiedfnc/OdM0u+uPgVKxzkere9IxkVNxgcUNSeNJEeZDw8/gxGdr4qEl6xgNekERB+1S+E5eV4Rf5mN1An7ZKQRNMOwLuubwVaqijXsh9gtBIGwQyaNFSBQ2maj0B42oywoLxJhDXYgwaDgjoNPz2pmzPYN4sy+RkNTx84WpemtWlSjJ/0pkOqih0za1TM5FAyNv60IWd50WUHJc/lsYyDXo89qTGORqCHf3zyBjw0U0XhxD7ZCLR6u8YLQBNMBE0= ceph-62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:02:31.117 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf6OmOd2AJDKBzcKyeXgHHHTikb/7kSocblHmTbIJuY9Vvts6Ri8Tafg7VzwP3W+WGF0v1QBqwZtKZH6EvSjlIqSu3/Fh9yIrgOeRXadfVNh0aiWlBCWNzs4FeedEVJcFKDE4PtMkyZxV6FxeS2pZ8Tzd270FO4JlfxWCwQMMEEfPReZdEdZkygTyGSCDeBGkVpRxl5ovFok4wZsGqRuijO+tYr0Ca92cO4aAZEoaIYYgbneNT+FvBr8t+/IzNoUBiedfnc/OdM0u+uPgVKxzkere9IxkVNxgcUNSeNJEeZDw8/gxGdr4qEl6xgNekERB+1S+E5eV4Rf5mN1An7ZKQRNMOwLuubwVaqijXsh9gtBIGwQyaNFSBQ2maj0B42oywoLxJhDXYgwaDgjoNPz2pmzPYN4sy+RkNTx84WpemtWlSjJ/0pkOqih0za1TM5FAyNv60IWd50WUHJc/lsYyDXo89qTGORqCHf3zyBjw0U0XhxD7ZCLR6u8YLQBNMBE0= ceph-62030760-9cd3-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-17T12:02:31.237 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf6OmOd2AJDKBzcKyeXgHHHTikb/7kSocblHmTbIJuY9Vvts6Ri8Tafg7VzwP3W+WGF0v1QBqwZtKZH6EvSjlIqSu3/Fh9yIrgOeRXadfVNh0aiWlBCWNzs4FeedEVJcFKDE4PtMkyZxV6FxeS2pZ8Tzd270FO4JlfxWCwQMMEEfPReZdEdZkygTyGSCDeBGkVpRxl5ovFok4wZsGqRuijO+tYr0Ca92cO4aAZEoaIYYgbneNT+FvBr8t+/IzNoUBiedfnc/OdM0u+uPgVKxzkere9IxkVNxgcUNSeNJEeZDw8/gxGdr4qEl6xgNekERB+1S+E5eV4Rf5mN1An7ZKQRNMOwLuubwVaqijXsh9gtBIGwQyaNFSBQ2maj0B42oywoLxJhDXYgwaDgjoNPz2pmzPYN4sy+RkNTx84WpemtWlSjJ/0pkOqih0za1TM5FAyNv60IWd50WUHJc/lsYyDXo89qTGORqCHf3zyBjw0U0XhxD7ZCLR6u8YLQBNMBE0= ceph-62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:02:31.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-17T12:02:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:02:33.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.998+0000 7f8538574700 1 -- 172.21.15.27:0/3117740174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 msgr2=0x7f8530101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:33.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.998+0000 7f8538574700 1 --2- 172.21.15.27:0/3117740174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 0x7f8530101340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f852c009910 tx=0x7f852c009c20 comp rx=0 tx=0).stop 2023-12-17T12:02:33.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.998+0000 7f8538574700 1 -- 172.21.15.27:0/3117740174 shutdown_connections 2023-12-17T12:02:33.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.998+0000 7f8538574700 1 --2- 172.21.15.27:0/3117740174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 0x7f8530101340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:33.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.998+0000 7f8538574700 1 -- 172.21.15.27:0/3117740174 >> 172.21.15.27:0/3117740174 conn(0x7f85300fc4f0 msgr2=0x7f85300fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:33.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.998+0000 7f8538574700 1 -- 172.21.15.27:0/3117740174 shutdown_connections 2023-12-17T12:02:33.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.998+0000 7f8538574700 1 -- 172.21.15.27:0/3117740174 wait complete. 2023-12-17T12:02:33.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.999+0000 7f8538574700 1 Processor -- start 2023-12-17T12:02:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.999+0000 7f8538574700 1 -- start start 2023-12-17T12:02:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.999+0000 7f8538574700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 0x7f8530198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:32.999+0000 7f8538574700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85301989d0 con 0x7f8530100f00 2023-12-17T12:02:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.000+0000 7f8536310700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 0x7f8530198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.000+0000 7f8536310700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 0x7f8530198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54374/0 (socket says 172.21.15.27:54374) 2023-12-17T12:02:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.000+0000 7f8536310700 1 -- 172.21.15.27:0/920954795 learned_addr learned my addr 172.21.15.27:0/920954795 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.000+0000 7f8536310700 1 -- 172.21.15.27:0/920954795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0095f0 con 0x7f8530100f00 2023-12-17T12:02:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.000+0000 7f8536310700 1 --2- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 0x7f8530198490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f852c005790 tx=0x7f852c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.001+0000 7f85277fe700 1 -- 172.21.15.27:0/920954795 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f852c00b960 con 0x7f8530100f00 2023-12-17T12:02:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.001+0000 7f85277fe700 1 -- 172.21.15.27:0/920954795 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f852c00f460 con 0x7f8530100f00 2023-12-17T12:02:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.001+0000 7f85277fe700 1 -- 172.21.15.27:0/920954795 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f852c0175d0 con 0x7f8530100f00 2023-12-17T12:02:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.001+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8530198c30 con 0x7f8530100f00 2023-12-17T12:02:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.001+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853019b900 con 0x7f8530100f00 2023-12-17T12:02:33.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.002+0000 7f85277fe700 1 -- 172.21.15.27:0/920954795 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f852c00f5d0 con 0x7f8530100f00 2023-12-17T12:02:33.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.002+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8530191e20 con 0x7f8530100f00 2023-12-17T12:02:33.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.003+0000 7f85277fe700 1 --2- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f851c038910 0x7f851c03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:33.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.003+0000 7f85277fe700 1 -- 172.21.15.27:0/920954795 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f852c04ca50 con 0x7f8530100f00 2023-12-17T12:02:33.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.003+0000 7f8535b0f700 1 --2- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f851c038910 0x7f851c03add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:33.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.004+0000 7f8535b0f700 1 --2- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f851c038910 0x7f851c03add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8520006fd0 tx=0x7f8520006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:33.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.008+0000 7f85277fe700 1 -- 172.21.15.27:0/920954795 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f852c02ab50 con 0x7f8530100f00 2023-12-17T12:02:33.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.223+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8530107350 con 0x7f8530100f00 2023-12-17T12:02:33.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.225+0000 7f85277fe700 1 -- 172.21.15.27:0/920954795 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f852c027070 con 0x7f8530100f00 2023-12-17T12:02:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f851c038910 msgr2=0x7f851c03add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 --2- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f851c038910 0x7f851c03add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8520006fd0 tx=0x7f8520006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 msgr2=0x7f8530198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 --2- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 0x7f8530198490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f852c005790 tx=0x7f852c0047b0 comp rx=0 tx=0).stop 2023-12-17T12:02:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 shutdown_connections 2023-12-17T12:02:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 --2- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f851c038910 0x7f851c03add0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 --2- 172.21.15.27:0/920954795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8530100f00 0x7f8530198490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 >> 172.21.15.27:0/920954795 conn(0x7f85300fc4f0 msgr2=0x7f85300fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 shutdown_connections 2023-12-17T12:02:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:33.229+0000 7f8538574700 1 -- 172.21.15.27:0/920954795 wait complete. 2023-12-17T12:02:33.633 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-17T12:02:33.633 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-17T12:02:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:02:34.510 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:34 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/920954795' entity='client.admin' 2023-12-17T12:02:35.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.432+0000 7eff3d0fd700 1 -- 172.21.15.27:0/988828259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 msgr2=0x7eff38103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.432+0000 7eff3d0fd700 1 --2- 172.21.15.27:0/988828259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 0x7eff38103de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7eff28009910 tx=0x7eff28009c20 comp rx=0 tx=0).stop 2023-12-17T12:02:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.433+0000 7eff3d0fd700 1 -- 172.21.15.27:0/988828259 shutdown_connections 2023-12-17T12:02:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.433+0000 7eff3d0fd700 1 --2- 172.21.15.27:0/988828259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 0x7eff38103de0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.433+0000 7eff3d0fd700 1 -- 172.21.15.27:0/988828259 >> 172.21.15.27:0/988828259 conn(0x7eff380fb400 msgr2=0x7eff380fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.433+0000 7eff3d0fd700 1 -- 172.21.15.27:0/988828259 shutdown_connections 2023-12-17T12:02:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.433+0000 7eff3d0fd700 1 -- 172.21.15.27:0/988828259 wait complete. 2023-12-17T12:02:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.434+0000 7eff3d0fd700 1 Processor -- start 2023-12-17T12:02:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.434+0000 7eff3d0fd700 1 -- start start 2023-12-17T12:02:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.434+0000 7eff3d0fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 0x7eff38101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.434+0000 7eff3d0fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff38101760 con 0x7eff381019f0 2023-12-17T12:02:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.434+0000 7eff36d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 0x7eff38101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.435+0000 7eff36d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 0x7eff38101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54382/0 (socket says 172.21.15.27:54382) 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.435+0000 7eff36d9d700 1 -- 172.21.15.27:0/1462856301 learned_addr learned my addr 172.21.15.27:0/1462856301 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.435+0000 7eff36d9d700 1 -- 172.21.15.27:0/1462856301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280095f0 con 0x7eff381019f0 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.435+0000 7eff36d9d700 1 --2- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 0x7eff38101220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff28005950 tx=0x7eff28005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.435+0000 7eff1ffff700 1 -- 172.21.15.27:0/1462856301 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff2800b960 con 0x7eff381019f0 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.435+0000 7eff1ffff700 1 -- 172.21.15.27:0/1462856301 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff2800f460 con 0x7eff381019f0 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.435+0000 7eff1ffff700 1 -- 172.21.15.27:0/1462856301 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff28017440 con 0x7eff381019f0 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.435+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff380ff9a0 con 0x7eff381019f0 2023-12-17T12:02:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.436+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff380ffe40 con 0x7eff381019f0 2023-12-17T12:02:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.437+0000 7eff1ffff700 1 -- 172.21.15.27:0/1462856301 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7eff2800f840 con 0x7eff381019f0 2023-12-17T12:02:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.437+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff38107460 con 0x7eff381019f0 2023-12-17T12:02:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.437+0000 7eff1ffff700 1 --2- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7eff20038910 0x7eff2003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.437+0000 7eff1ffff700 1 -- 172.21.15.27:0/1462856301 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff2804cb20 con 0x7eff381019f0 2023-12-17T12:02:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.437+0000 7eff3659c700 1 --2- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7eff20038910 0x7eff2003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.438+0000 7eff3659c700 1 --2- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7eff20038910 0x7eff2003add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006fd0 tx=0x7eff2c006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.442+0000 7eff1ffff700 1 -- 172.21.15.27:0/1462856301 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7eff2800be40 con 0x7eff381019f0 2023-12-17T12:02:35.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.657+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7eff38100080 con 0x7eff20038910 2023-12-17T12:02:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.661+0000 7eff1ffff700 1 -- 172.21.15.27:0/1462856301 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7eff38100080 con 0x7eff20038910 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7eff20038910 msgr2=0x7eff2003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 --2- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7eff20038910 0x7eff2003add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006fd0 tx=0x7eff2c006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 msgr2=0x7eff38101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 --2- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 0x7eff38101220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff28005950 tx=0x7eff28005810 comp rx=0 tx=0).stop 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 shutdown_connections 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 --2- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7eff20038910 0x7eff2003add0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 --2- 172.21.15.27:0/1462856301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff381019f0 0x7eff38101220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 >> 172.21.15.27:0/1462856301 conn(0x7eff380fb400 msgr2=0x7eff380fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 shutdown_connections 2023-12-17T12:02:35.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:35.663+0000 7eff3d0fd700 1 -- 172.21.15.27:0/1462856301 wait complete. 2023-12-17T12:02:36.079 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi141 2023-12-17T12:02:36.079 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T12:02:36.080 DEBUG:teuthology.orchestra.run.smithi141:> dd of=/etc/ceph/ceph.conf 2023-12-17T12:02:36.128 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T12:02:36.128 DEBUG:teuthology.orchestra.run.smithi141:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:02:36.213 INFO:tasks.cephadm:Adding host smithi141 to orchestrator... 2023-12-17T12:02:36.213 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch host add smithi141 2023-12-17T12:02:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:02:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:02:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:36.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:36.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:02:36.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:02:36.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:02:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:02:37.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:37 smithi027 ceph-mon[36871]: Updating smithi027:/etc/ceph/ceph.conf 2023-12-17T12:02:37.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:37 smithi027 ceph-mon[36871]: Updating smithi027:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:02:37.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.867+0000 7f4129dc2700 1 -- 172.21.15.27:0/3961143406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 msgr2=0x7f4124073de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:37.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.867+0000 7f4129dc2700 1 --2- 172.21.15.27:0/3961143406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 0x7f4124073de0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4114009910 tx=0x7f4114009c20 comp rx=0 tx=0).stop 2023-12-17T12:02:37.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.867+0000 7f4129dc2700 1 -- 172.21.15.27:0/3961143406 shutdown_connections 2023-12-17T12:02:37.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.867+0000 7f4129dc2700 1 --2- 172.21.15.27:0/3961143406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 0x7f4124073de0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:37.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.867+0000 7f4129dc2700 1 -- 172.21.15.27:0/3961143406 >> 172.21.15.27:0/3961143406 conn(0x7f41240fc9d0 msgr2=0x7f41240fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:37.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.867+0000 7f4129dc2700 1 -- 172.21.15.27:0/3961143406 shutdown_connections 2023-12-17T12:02:37.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.867+0000 7f4129dc2700 1 -- 172.21.15.27:0/3961143406 wait complete. 2023-12-17T12:02:37.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.868+0000 7f4129dc2700 1 Processor -- start 2023-12-17T12:02:37.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.868+0000 7f4129dc2700 1 -- start start 2023-12-17T12:02:37.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.868+0000 7f4129dc2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 0x7f412419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:37.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.869+0000 7f4129dc2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412419eb90 con 0x7f4124073a00 2023-12-17T12:02:37.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.869+0000 7f41237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 0x7f412419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:37.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.869+0000 7f41237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 0x7f412419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50842/0 (socket says 172.21.15.27:50842) 2023-12-17T12:02:37.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.869+0000 7f41237fe700 1 -- 172.21.15.27:0/1738922526 learned_addr learned my addr 172.21.15.27:0/1738922526 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.869+0000 7f41237fe700 1 -- 172.21.15.27:0/1738922526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140095f0 con 0x7f4124073a00 2023-12-17T12:02:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.869+0000 7f41237fe700 1 --2- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 0x7f412419e650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41140098e0 tx=0x7f4114005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.870+0000 7f4120ff9700 1 -- 172.21.15.27:0/1738922526 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f411401d070 con 0x7f4124073a00 2023-12-17T12:02:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.870+0000 7f4120ff9700 1 -- 172.21.15.27:0/1738922526 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4114009e60 con 0x7f4124073a00 2023-12-17T12:02:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.870+0000 7f4120ff9700 1 -- 172.21.15.27:0/1738922526 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4114022c10 con 0x7f4124073a00 2023-12-17T12:02:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.870+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412419ed30 con 0x7f4124073a00 2023-12-17T12:02:37.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.870+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412419f150 con 0x7f4124073a00 2023-12-17T12:02:37.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.871+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41240750b0 con 0x7f4124073a00 2023-12-17T12:02:37.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.871+0000 7f4120ff9700 1 -- 172.21.15.27:0/1738922526 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f4114022470 con 0x7f4124073a00 2023-12-17T12:02:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.871+0000 7f4120ff9700 1 --2- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f410c038910 0x7f410c03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.872+0000 7f4120ff9700 1 -- 172.21.15.27:0/1738922526 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f411404ca60 con 0x7f4124073a00 2023-12-17T12:02:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.872+0000 7f4122ffd700 1 --2- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f410c038910 0x7f410c03add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:37.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.872+0000 7f4122ffd700 1 --2- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f410c038910 0x7f410c03add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4118006fd0 tx=0x7f4118006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:37.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:37.877+0000 7f4120ff9700 1 -- 172.21.15.27:0/1738922526 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f411402b980 con 0x7f4124073a00 2023-12-17T12:02:38.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:38.094+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi141", "target": ["mon-mgr", ""]}) v1 -- 0x7f412410d370 con 0x7f410c038910 2023-12-17T12:02:38.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:38 smithi027 ceph-mon[36871]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:02:38.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:38 smithi027 ceph-mon[36871]: Updating smithi027:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-17T12:02:38.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:38 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:38.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:38 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:38.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:38 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:39.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:39 smithi027 ceph-mon[36871]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi141", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:40.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:40 smithi027 ceph-mon[36871]: Deploying cephadm binary to smithi141 2023-12-17T12:02:42.996 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi141' with addr '172.21.15.141' 2023-12-17T12:02:42.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.993+0000 7f4120ff9700 1 -- 172.21.15.27:0/1738922526 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f412410d370 con 0x7f410c038910 2023-12-17T12:02:42.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.996+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f410c038910 msgr2=0x7f410c03add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:42.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.996+0000 7f4129dc2700 1 --2- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f410c038910 0x7f410c03add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4118006fd0 tx=0x7f4118006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:42.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.996+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 msgr2=0x7f412419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.996+0000 7f4129dc2700 1 --2- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 0x7f412419e650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41140098e0 tx=0x7f4114005710 comp rx=0 tx=0).stop 2023-12-17T12:02:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.997+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 shutdown_connections 2023-12-17T12:02:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.997+0000 7f4129dc2700 1 --2- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f410c038910 0x7f410c03add0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.997+0000 7f4129dc2700 1 --2- 172.21.15.27:0/1738922526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124073a00 0x7f412419e650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.997+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 >> 172.21.15.27:0/1738922526 conn(0x7f41240fc9d0 msgr2=0x7f4124107190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.997+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 shutdown_connections 2023-12-17T12:02:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:42.997+0000 7f4129dc2700 1 -- 172.21.15.27:0/1738922526 wait complete. 2023-12-17T12:02:43.376 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch host ls --format=json 2023-12-17T12:02:43.808 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:02:44.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:43 smithi027 ceph-mon[36871]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:02:44.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:44.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:43 smithi027 ceph-mon[36871]: Added host smithi141 2023-12-17T12:02:44.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:02:44.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.927+0000 7f5e5cd04700 1 -- 172.21.15.27:0/3912807949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 msgr2=0x7f5e58101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.927+0000 7f5e5cd04700 1 --2- 172.21.15.27:0/3912807949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 0x7f5e58101520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009910 tx=0x7f5e48009c20 comp rx=0 tx=0).stop 2023-12-17T12:02:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.927+0000 7f5e5cd04700 1 -- 172.21.15.27:0/3912807949 shutdown_connections 2023-12-17T12:02:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.927+0000 7f5e5cd04700 1 --2- 172.21.15.27:0/3912807949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 0x7f5e58101520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.927+0000 7f5e5cd04700 1 -- 172.21.15.27:0/3912807949 >> 172.21.15.27:0/3912807949 conn(0x7f5e580fc9d0 msgr2=0x7f5e580fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.927+0000 7f5e5cd04700 1 -- 172.21.15.27:0/3912807949 shutdown_connections 2023-12-17T12:02:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.927+0000 7f5e5cd04700 1 -- 172.21.15.27:0/3912807949 wait complete. 2023-12-17T12:02:44.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.928+0000 7f5e5cd04700 1 Processor -- start 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.929+0000 7f5e5cd04700 1 -- start start 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.929+0000 7f5e5cd04700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.929+0000 7f5e5cd04700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5819eb70 con 0x7f5e58101140 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.929+0000 7f5e5659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.929+0000 7f5e5659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50866/0 (socket says 172.21.15.27:50866) 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.929+0000 7f5e5659c700 1 -- 172.21.15.27:0/1411653343 learned_addr learned my addr 172.21.15.27:0/1411653343 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.930+0000 7f5e5659c700 1 -- 172.21.15.27:0/1411653343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480095f0 con 0x7f5e58101140 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.930+0000 7f5e5659c700 1 --2- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e480098e0 tx=0x7f5e48005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.930+0000 7f5e477fe700 1 -- 172.21.15.27:0/1411653343 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e4801d070 con 0x7f5e58101140 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.930+0000 7f5e477fe700 1 -- 172.21.15.27:0/1411653343 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e48009e60 con 0x7f5e58101140 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.930+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5819ed70 con 0x7f5e58101140 2023-12-17T12:02:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.930+0000 7f5e477fe700 1 -- 172.21.15.27:0/1411653343 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e48022b30 con 0x7f5e58101140 2023-12-17T12:02:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.930+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5819f190 con 0x7f5e58101140 2023-12-17T12:02:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.931+0000 7f5e477fe700 1 -- 172.21.15.27:0/1411653343 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f5e48022470 con 0x7f5e58101140 2023-12-17T12:02:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.931+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e58102620 con 0x7f5e58101140 2023-12-17T12:02:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.932+0000 7f5e477fe700 1 --2- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5e40038910 0x7f5e4003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.932+0000 7f5e477fe700 1 -- 172.21.15.27:0/1411653343 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5e4804cdf0 con 0x7f5e58101140 2023-12-17T12:02:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.932+0000 7f5e55d9b700 1 --2- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5e40038910 0x7f5e4003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.933+0000 7f5e55d9b700 1 --2- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5e40038910 0x7f5e4003add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c006fd0 tx=0x7f5e4c006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:44.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:44.937+0000 7f5e477fe700 1 -- 172.21.15.27:0/1411653343 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5e4802ba00 con 0x7f5e58101140 2023-12-17T12:02:45.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.157+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e5810b0e0 con 0x7f5e40038910 2023-12-17T12:02:45.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.158+0000 7f5e477fe700 1 -- 172.21.15.27:0/1411653343 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f5e5810b0e0 con 0x7f5e40038910 2023-12-17T12:02:45.161 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:45.161 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.141", "hostname": "smithi141", "labels": [], "status": ""}] 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5e40038910 msgr2=0x7f5e4003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 --2- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5e40038910 0x7f5e4003add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c006fd0 tx=0x7f5e4c006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 msgr2=0x7f5e5819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 --2- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e480098e0 tx=0x7f5e48005710 comp rx=0 tx=0).stop 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 shutdown_connections 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 --2- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5e40038910 0x7f5e4003add0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 --2- 172.21.15.27:0/1411653343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 >> 172.21.15.27:0/1411653343 conn(0x7f5e580fc9d0 msgr2=0x7f5e58073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 shutdown_connections 2023-12-17T12:02:45.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:45.161+0000 7f5e5cd04700 1 -- 172.21.15.27:0/1411653343 wait complete. 2023-12-17T12:02:45.348 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:45 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:45.534 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2023-12-17T12:02:45.534 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T12:02:45.534 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2023-12-17T12:02:45.583 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T12:02:45.583 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:02:45.668 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2023-12-17T12:02:45.668 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch host add smithi192 2023-12-17T12:02:46.088 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:02:46.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:46 smithi027 ceph-mon[36871]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:02:46.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:46 smithi027 ceph-mon[36871]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:02:46.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:46 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:47.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.245+0000 7f47f96b6700 1 -- 172.21.15.27:0/963435153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 msgr2=0x7f47f4107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:47.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.245+0000 7f47f96b6700 1 --2- 172.21.15.27:0/963435153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 0x7f47f4107360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009910 tx=0x7f47e4009c20 comp rx=0 tx=0).stop 2023-12-17T12:02:47.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.245+0000 7f47f96b6700 1 -- 172.21.15.27:0/963435153 shutdown_connections 2023-12-17T12:02:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.245+0000 7f47f96b6700 1 --2- 172.21.15.27:0/963435153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 0x7f47f4107360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.245+0000 7f47f96b6700 1 -- 172.21.15.27:0/963435153 >> 172.21.15.27:0/963435153 conn(0x7f47f4075930 msgr2=0x7f47f4075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.245+0000 7f47f96b6700 1 -- 172.21.15.27:0/963435153 shutdown_connections 2023-12-17T12:02:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.245+0000 7f47f96b6700 1 -- 172.21.15.27:0/963435153 wait complete. 2023-12-17T12:02:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.246+0000 7f47f96b6700 1 Processor -- start 2023-12-17T12:02:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.246+0000 7f47f96b6700 1 -- start start 2023-12-17T12:02:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.247+0000 7f47f96b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 0x7f47f40fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.247+0000 7f47f96b6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f4101e10 con 0x7f47f4106f80 2023-12-17T12:02:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.247+0000 7f47f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 0x7f47f40fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.247+0000 7f47f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 0x7f47f40fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36616/0 (socket says 172.21.15.27:36616) 2023-12-17T12:02:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.247+0000 7f47f2ffd700 1 -- 172.21.15.27:0/1375533966 learned_addr learned my addr 172.21.15.27:0/1375533966 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.247+0000 7f47f2ffd700 1 -- 172.21.15.27:0/1375533966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e40095f0 con 0x7f47f4106f80 2023-12-17T12:02:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.248+0000 7f47f2ffd700 1 --2- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 0x7f47f40fff00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47e40098e0 tx=0x7f47e4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.248+0000 7f47dbfff700 1 -- 172.21.15.27:0/1375533966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47e401d070 con 0x7f47f4106f80 2023-12-17T12:02:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.248+0000 7f47dbfff700 1 -- 172.21.15.27:0/1375533966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47e4009e60 con 0x7f47f4106f80 2023-12-17T12:02:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.248+0000 7f47dbfff700 1 -- 172.21.15.27:0/1375533966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47e4022b30 con 0x7f47f4106f80 2023-12-17T12:02:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.248+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f4100440 con 0x7f47f4106f80 2023-12-17T12:02:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.248+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f41008c0 con 0x7f47f4106f80 2023-12-17T12:02:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.249+0000 7f47dbfff700 1 -- 172.21.15.27:0/1375533966 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f47e4022470 con 0x7f47f4106f80 2023-12-17T12:02:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.249+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f4102630 con 0x7f47f4106f80 2023-12-17T12:02:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.250+0000 7f47dbfff700 1 --2- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47dc038960 0x7f47dc03ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.250+0000 7f47dbfff700 1 -- 172.21.15.27:0/1375533966 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f47e404cdf0 con 0x7f47f4106f80 2023-12-17T12:02:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.250+0000 7f47f27fc700 1 --2- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47dc038960 0x7f47dc03ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.250+0000 7f47f27fc700 1 --2- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47dc038960 0x7f47dc03ae20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47e8006fd0 tx=0x7f47e8006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:47.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.254+0000 7f47dbfff700 1 -- 172.21.15.27:0/1375533966 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f47e402ba00 con 0x7f47f4106f80 2023-12-17T12:02:47.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:47.469+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7f47f4100b70 con 0x7f47dc038960 2023-12-17T12:02:48.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:48 smithi027 ceph-mon[36871]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:02:48.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:48 smithi027 ceph-mon[36871]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:02:49.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:49 smithi027 ceph-mon[36871]: Deploying cephadm binary to smithi192 2023-12-17T12:02:50.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:50 smithi027 ceph-mon[36871]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:02:52.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.246+0000 7f47dbfff700 1 -- 172.21.15.27:0/1375533966 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f47f4100b70 con 0x7f47dc038960 2023-12-17T12:02:52.248 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi192' with addr '172.21.15.192' 2023-12-17T12:02:52.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.248+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47dc038960 msgr2=0x7f47dc03ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:52.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.248+0000 7f47f96b6700 1 --2- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47dc038960 0x7f47dc03ae20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47e8006fd0 tx=0x7f47e8006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:52.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.248+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 msgr2=0x7f47f40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:52.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.248+0000 7f47f96b6700 1 --2- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 0x7f47f40fff00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47e40098e0 tx=0x7f47e4005710 comp rx=0 tx=0).stop 2023-12-17T12:02:52.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.248+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 shutdown_connections 2023-12-17T12:02:52.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.248+0000 7f47f96b6700 1 --2- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47dc038960 0x7f47dc03ae20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:52.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.248+0000 7f47f96b6700 1 --2- 172.21.15.27:0/1375533966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f4106f80 0x7f47f40fff00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:52.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.248+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 >> 172.21.15.27:0/1375533966 conn(0x7f47f4075930 msgr2=0x7f47f40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:52.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.249+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 shutdown_connections 2023-12-17T12:02:52.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:52.249+0000 7f47f96b6700 1 -- 172.21.15.27:0/1375533966 wait complete. 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi141", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: Updating smithi141:/etc/ceph/ceph.conf 2023-12-17T12:02:52.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:52.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch host ls --format=json 2023-12-17T12:02:53.042 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:02:53.523 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:53 smithi027 ceph-mon[36871]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:02:53.523 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:53 smithi027 ceph-mon[36871]: Added host smithi192 2023-12-17T12:02:53.523 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:53 smithi027 ceph-mon[36871]: Updating smithi141:/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:02:54.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.234+0000 7f70a0e6e700 1 -- 172.21.15.27:0/1960352672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 msgr2=0x7f709c073de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:54.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.234+0000 7f70a0e6e700 1 --2- 172.21.15.27:0/1960352672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 0x7f709c073de0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f708c009910 tx=0x7f708c009c20 comp rx=0 tx=0).stop 2023-12-17T12:02:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.235+0000 7f70a0e6e700 1 -- 172.21.15.27:0/1960352672 shutdown_connections 2023-12-17T12:02:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.235+0000 7f70a0e6e700 1 --2- 172.21.15.27:0/1960352672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 0x7f709c073de0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.235+0000 7f70a0e6e700 1 -- 172.21.15.27:0/1960352672 >> 172.21.15.27:0/1960352672 conn(0x7f709c0fc7e0 msgr2=0x7f709c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.235+0000 7f70a0e6e700 1 -- 172.21.15.27:0/1960352672 shutdown_connections 2023-12-17T12:02:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.235+0000 7f70a0e6e700 1 -- 172.21.15.27:0/1960352672 wait complete. 2023-12-17T12:02:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.236+0000 7f70a0e6e700 1 Processor -- start 2023-12-17T12:02:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.236+0000 7f70a0e6e700 1 -- start start 2023-12-17T12:02:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.237+0000 7f70a0e6e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 0x7f709c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.237+0000 7f70a0e6e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c198cd0 con 0x7f709c073a00 2023-12-17T12:02:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.237+0000 7f709b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 0x7f709c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.237+0000 7f709b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 0x7f709c198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36642/0 (socket says 172.21.15.27:36642) 2023-12-17T12:02:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.237+0000 7f709b7fe700 1 -- 172.21.15.27:0/66133375 learned_addr learned my addr 172.21.15.27:0/66133375 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.237+0000 7f709b7fe700 1 -- 172.21.15.27:0/66133375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0095f0 con 0x7f709c073a00 2023-12-17T12:02:54.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.238+0000 7f709b7fe700 1 --2- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 0x7f709c198790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f708c0055f0 tx=0x7f708c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.238+0000 7f7098ff9700 1 -- 172.21.15.27:0/66133375 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f708c01d070 con 0x7f709c073a00 2023-12-17T12:02:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.238+0000 7f7098ff9700 1 -- 172.21.15.27:0/66133375 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f708c00b750 con 0x7f709c073a00 2023-12-17T12:02:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.238+0000 7f7098ff9700 1 -- 172.21.15.27:0/66133375 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f708c0037b0 con 0x7f709c073a00 2023-12-17T12:02:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.238+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c198ed0 con 0x7f709c073a00 2023-12-17T12:02:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.238+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c1992f0 con 0x7f709c073a00 2023-12-17T12:02:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.239+0000 7f7098ff9700 1 -- 172.21.15.27:0/66133375 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f708c004390 con 0x7f709c073a00 2023-12-17T12:02:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.239+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c10a3b0 con 0x7f709c073a00 2023-12-17T12:02:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.240+0000 7f7098ff9700 1 --2- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7084038910 0x7f708403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.240+0000 7f7098ff9700 1 -- 172.21.15.27:0/66133375 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f708c04c670 con 0x7f709c073a00 2023-12-17T12:02:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.240+0000 7f709affd700 1 --2- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7084038910 0x7f708403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.241+0000 7f709affd700 1 --2- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7084038910 0x7f708403add0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7090006fd0 tx=0x7f7090006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.245+0000 7f7098ff9700 1 -- 172.21.15.27:0/66133375 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f708c00b8c0 con 0x7f709c073a00 2023-12-17T12:02:54.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.461+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f709c10d130 con 0x7f7084038910 2023-12-17T12:02:54.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.462+0000 7f7098ff9700 1 -- 172.21.15.27:0/66133375 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f709c10d130 con 0x7f7084038910 2023-12-17T12:02:54.464 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:02:54.464 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.141", "hostname": "smithi141", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2023-12-17T12:02:54.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7084038910 msgr2=0x7f708403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:54.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 --2- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7084038910 0x7f708403add0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7090006fd0 tx=0x7f7090006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:54.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 msgr2=0x7f709c198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:54.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 --2- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 0x7f709c198790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f708c0055f0 tx=0x7f708c0056d0 comp rx=0 tx=0).stop 2023-12-17T12:02:54.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 shutdown_connections 2023-12-17T12:02:54.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 --2- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7084038910 0x7f708403add0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:54.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 --2- 172.21.15.27:0/66133375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f709c073a00 0x7f709c198790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:54.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 >> 172.21.15.27:0/66133375 conn(0x7f709c0fc7e0 msgr2=0x7f709c106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:54.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 shutdown_connections 2023-12-17T12:02:54.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:54.465+0000 7f70a0e6e700 1 -- 172.21.15.27:0/66133375 wait complete. 2023-12-17T12:02:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:54 smithi027 ceph-mon[36871]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:02:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:54 smithi027 ceph-mon[36871]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-17T12:02:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:02:54.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:54.896 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-17T12:02:54.897 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd crush tunables default 2023-12-17T12:02:55.314 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:02:55.797 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:55 smithi027 ceph-mon[36871]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:02:55.798 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:56.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.451+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/2317165437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 msgr2=0x7f3b48107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.451+0000 7f3b4f0cb700 1 --2- 172.21.15.27:0/2317165437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 0x7f3b48107360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009910 tx=0x7f3b3c009c20 comp rx=0 tx=0).stop 2023-12-17T12:02:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.452+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/2317165437 shutdown_connections 2023-12-17T12:02:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.452+0000 7f3b4f0cb700 1 --2- 172.21.15.27:0/2317165437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 0x7f3b48107360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.452+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/2317165437 >> 172.21.15.27:0/2317165437 conn(0x7f3b48075930 msgr2=0x7f3b48075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.452+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/2317165437 shutdown_connections 2023-12-17T12:02:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.452+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/2317165437 wait complete. 2023-12-17T12:02:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.453+0000 7f3b4f0cb700 1 Processor -- start 2023-12-17T12:02:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.453+0000 7f3b4f0cb700 1 -- start start 2023-12-17T12:02:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.453+0000 7f3b4f0cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 0x7f3b4819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.453+0000 7f3b4f0cb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4819ebe0 con 0x7f3b48106f80 2023-12-17T12:02:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b4ce67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 0x7f3b4819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b4ce67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 0x7f3b4819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51892/0 (socket says 172.21.15.27:51892) 2023-12-17T12:02:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b4ce67700 1 -- 172.21.15.27:0/675045840 learned_addr learned my addr 172.21.15.27:0/675045840 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:02:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b4ce67700 1 -- 172.21.15.27:0/675045840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c0095f0 con 0x7f3b48106f80 2023-12-17T12:02:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b4ce67700 1 --2- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 0x7f3b4819e6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005660 tx=0x7f3b3c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b45ffb700 1 -- 172.21.15.27:0/675045840 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b3c01d070 con 0x7f3b48106f80 2023-12-17T12:02:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b45ffb700 1 -- 172.21.15.27:0/675045840 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b3c00b6f0 con 0x7f3b48106f80 2023-12-17T12:02:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b45ffb700 1 -- 172.21.15.27:0/675045840 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b3c0037f0 con 0x7f3b48106f80 2023-12-17T12:02:56.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.454+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4819ee40 con 0x7f3b48106f80 2023-12-17T12:02:56.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.455+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b48198750 con 0x7f3b48106f80 2023-12-17T12:02:56.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.456+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b48102630 con 0x7f3b48106f80 2023-12-17T12:02:56.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.456+0000 7f3b45ffb700 1 -- 172.21.15.27:0/675045840 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f3b3c00b860 con 0x7f3b48106f80 2023-12-17T12:02:56.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.456+0000 7f3b45ffb700 1 --2- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f3b38038910 0x7f3b3803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:56.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.456+0000 7f3b45ffb700 1 -- 172.21.15.27:0/675045840 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3b3c04c5d0 con 0x7f3b48106f80 2023-12-17T12:02:56.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.456+0000 7f3b47fff700 1 --2- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f3b38038910 0x7f3b3803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:56.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.457+0000 7f3b47fff700 1 --2- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f3b38038910 0x7f3b3803add0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006fd0 tx=0x7f3b34006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:56.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.461+0000 7f3b45ffb700 1 -- 172.21.15.27:0/675045840 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3b3c02ad40 con 0x7f3b48106f80 2023-12-17T12:02:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:56.680+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3b48199c30 con 0x7f3b48106f80 2023-12-17T12:02:56.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:56 smithi027 ceph-mon[36871]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:02:56.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:02:57.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.568+0000 7f3b45ffb700 1 -- 172.21.15.27:0/675045840 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3b3c026020 con 0x7f3b48106f80 2023-12-17T12:02:57.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f3b38038910 msgr2=0x7f3b3803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:57.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 --2- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f3b38038910 0x7f3b3803add0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006fd0 tx=0x7f3b34006e40 comp rx=0 tx=0).stop 2023-12-17T12:02:57.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 msgr2=0x7f3b4819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:57.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 --2- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 0x7f3b4819e6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005660 tx=0x7f3b3c005740 comp rx=0 tx=0).stop 2023-12-17T12:02:57.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 shutdown_connections 2023-12-17T12:02:57.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 --2- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f3b38038910 0x7f3b3803add0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:57.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 --2- 172.21.15.27:0/675045840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b48106f80 0x7f3b4819e6a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:57.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 >> 172.21.15.27:0/675045840 conn(0x7f3b48075930 msgr2=0x7f3b481033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:57.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 shutdown_connections 2023-12-17T12:02:57.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:02:57.572+0000 7f3b4f0cb700 1 -- 172.21.15.27:0/675045840 wait complete. 2023-12-17T12:02:57.575 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2023-12-17T12:02:57.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:57 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/675045840' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-17T12:02:57.954 INFO:tasks.cephadm:Adding mon.a on smithi027 2023-12-17T12:02:57.954 INFO:tasks.cephadm:Adding mon.b on smithi141 2023-12-17T12:02:57.954 INFO:tasks.cephadm:Adding mon.c on smithi192 2023-12-17T12:02:57.955 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch apply mon '3;smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c' 2023-12-17T12:02:58.366 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2023-12-17T12:02:58.466 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2023-12-17T12:02:58.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:58 smithi027 ceph-mon[36871]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:02:58.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:58 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/675045840' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-17T12:02:58.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:02:58 smithi027 ceph-mon[36871]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:02:59.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.914+0000 7f451d4c7700 1 -- 172.21.15.192:0/636114304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 msgr2=0x7f4518073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:02:59.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.914+0000 7f451d4c7700 1 --2- 172.21.15.192:0/636114304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 0x7f4518073a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4508009910 tx=0x7f4508009c20 comp rx=0 tx=0).stop 2023-12-17T12:02:59.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.914+0000 7f451d4c7700 1 -- 172.21.15.192:0/636114304 shutdown_connections 2023-12-17T12:02:59.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.914+0000 7f451d4c7700 1 --2- 172.21.15.192:0/636114304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 0x7f4518073a60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:02:59.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.914+0000 7f451d4c7700 1 -- 172.21.15.192:0/636114304 >> 172.21.15.192:0/636114304 conn(0x7f45180fc4f0 msgr2=0x7f45180fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:02:59.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.915+0000 7f451d4c7700 1 -- 172.21.15.192:0/636114304 shutdown_connections 2023-12-17T12:02:59.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.915+0000 7f451d4c7700 1 -- 172.21.15.192:0/636114304 wait complete. 2023-12-17T12:02:59.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.916+0000 7f451d4c7700 1 Processor -- start 2023-12-17T12:02:59.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.916+0000 7f451d4c7700 1 -- start start 2023-12-17T12:02:59.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.916+0000 7f451d4c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 0x7f4518198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:59.919 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.916+0000 7f451d4c7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4518198a40 con 0x7f4518075600 2023-12-17T12:02:59.919 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.916+0000 7f4516ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 0x7f4518198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:59.919 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.916+0000 7f4516ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 0x7f4518198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.192:32974/0 (socket says 172.21.15.192:32974) 2023-12-17T12:02:59.919 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.916+0000 7f4516ffd700 1 -- 172.21.15.192:0/1870853385 learned_addr learned my addr 172.21.15.192:0/1870853385 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:02:59.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.917+0000 7f4516ffd700 1 -- 172.21.15.192:0/1870853385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080095f0 con 0x7f4518075600 2023-12-17T12:02:59.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.917+0000 7f4516ffd700 1 --2- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 0x7f4518198500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45080098e0 tx=0x7f4508005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:59.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.918+0000 7f44fffff700 1 -- 172.21.15.192:0/1870853385 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f450800ba90 con 0x7f4518075600 2023-12-17T12:02:59.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.918+0000 7f44fffff700 1 -- 172.21.15.192:0/1870853385 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f450800bbf0 con 0x7f4518075600 2023-12-17T12:02:59.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.918+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4518198c40 con 0x7f4518075600 2023-12-17T12:02:59.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.918+0000 7f44fffff700 1 -- 172.21.15.192:0/1870853385 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f450800fe70 con 0x7f4518075600 2023-12-17T12:02:59.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.918+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45181990e0 con 0x7f4518075600 2023-12-17T12:02:59.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.919+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4518191d30 con 0x7f4518075600 2023-12-17T12:02:59.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.919+0000 7f44fffff700 1 -- 172.21.15.192:0/1870853385 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f450800f460 con 0x7f4518075600 2023-12-17T12:02:59.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.919+0000 7f44fffff700 1 --2- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4500038910 0x7f450003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:02:59.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.919+0000 7f44fffff700 1 -- 172.21.15.192:0/1870853385 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f450804cab0 con 0x7f4518075600 2023-12-17T12:02:59.922 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.920+0000 7f45167fc700 1 --2- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4500038910 0x7f450003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:02:59.922 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.921+0000 7f45167fc700 1 --2- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4500038910 0x7f450003add0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f450c006fd0 tx=0x7f450c006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:02:59.927 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:02:59.925+0000 7f44fffff700 1 -- 172.21.15.192:0/1870853385 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f450802ad30 con 0x7f4518075600 2023-12-17T12:03:00.175 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.173+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f4518061ad0 con 0x7f4500038910 2023-12-17T12:03:00.182 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.180+0000 7f44fffff700 1 -- 172.21.15.192:0/1870853385 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4518061ad0 con 0x7f4500038910 2023-12-17T12:03:00.182 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2023-12-17T12:03:00.184 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4500038910 msgr2=0x7f450003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:00.184 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 --2- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4500038910 0x7f450003add0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f450c006fd0 tx=0x7f450c006e40 comp rx=0 tx=0).stop 2023-12-17T12:03:00.184 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 msgr2=0x7f4518198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:00.185 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 --2- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 0x7f4518198500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45080098e0 tx=0x7f4508005810 comp rx=0 tx=0).stop 2023-12-17T12:03:00.185 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 shutdown_connections 2023-12-17T12:03:00.185 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 --2- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4500038910 0x7f450003add0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:00.185 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 --2- 172.21.15.192:0/1870853385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4518075600 0x7f4518198500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:00.185 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 >> 172.21.15.192:0/1870853385 conn(0x7f45180fc4f0 msgr2=0x7f45180fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:00.185 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.182+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 shutdown_connections 2023-12-17T12:03:00.185 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:00.183+0000 7f451d4c7700 1 -- 172.21.15.192:0/1870853385 wait complete. 2023-12-17T12:03:00.661 DEBUG:teuthology.orchestra.run.smithi141:mon.b> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.b.service 2023-12-17T12:03:00.666 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.c.service 2023-12-17T12:03:00.671 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-17T12:03:00.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-17T12:03:00.743 INFO:journalctl@ceph.mon.b.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:03:00.746 INFO:journalctl@ceph.mon.c.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: Updating smithi192:/etc/ceph/ceph.conf 2023-12-17T12:03:00.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:01.118 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:01.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:01 smithi027 ceph-mon[36871]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:01.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:01 smithi027 ceph-mon[36871]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c;count:3 2023-12-17T12:03:01.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:01 smithi027 ceph-mon[36871]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:02.310 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.307+0000 7f9c9f41e700 1 -- 172.21.15.192:0/891899582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 msgr2=0x7f9c98068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:02.310 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.307+0000 7f9c9f41e700 1 --2- 172.21.15.192:0/891899582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 0x7f9c98068af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009910 tx=0x7f9c94009c20 comp rx=0 tx=0).stop 2023-12-17T12:03:02.310 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.307+0000 7f9c9f41e700 1 -- 172.21.15.192:0/891899582 shutdown_connections 2023-12-17T12:03:02.310 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.307+0000 7f9c9f41e700 1 --2- 172.21.15.192:0/891899582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 0x7f9c98068af0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:02.310 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.307+0000 7f9c9f41e700 1 -- 172.21.15.192:0/891899582 >> 172.21.15.192:0/891899582 conn(0x7f9c98075930 msgr2=0x7f9c98075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:02.310 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.307+0000 7f9c9f41e700 1 -- 172.21.15.192:0/891899582 shutdown_connections 2023-12-17T12:03:02.310 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.307+0000 7f9c9f41e700 1 -- 172.21.15.192:0/891899582 wait complete. 2023-12-17T12:03:02.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.308+0000 7f9c9f41e700 1 Processor -- start 2023-12-17T12:03:02.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.309+0000 7f9c9f41e700 1 -- start start 2023-12-17T12:03:02.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.309+0000 7f9c9f41e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 0x7f9c9819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:02.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.309+0000 7f9c9f41e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819eb10 con 0x7f9c98068710 2023-12-17T12:03:02.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.309+0000 7f9c9d1ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 0x7f9c9819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:02.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.309+0000 7f9c9d1ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 0x7f9c9819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.192:32990/0 (socket says 172.21.15.192:32990) 2023-12-17T12:03:02.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.309+0000 7f9c9d1ba700 1 -- 172.21.15.192:0/3674488231 learned_addr learned my addr 172.21.15.192:0/3674488231 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:03:02.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.310+0000 7f9c9d1ba700 1 -- 172.21.15.192:0/3674488231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940095f0 con 0x7f9c98068710 2023-12-17T12:03:02.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.310+0000 7f9c9d1ba700 1 --2- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 0x7f9c9819e5d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c940098e0 tx=0x7f9c94005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:02.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.310+0000 7f9c8e7fc700 1 -- 172.21.15.192:0/3674488231 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c9401d070 con 0x7f9c98068710 2023-12-17T12:03:02.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.310+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9819ed10 con 0x7f9c98068710 2023-12-17T12:03:02.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.310+0000 7f9c8e7fc700 1 -- 172.21.15.192:0/3674488231 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c94009e60 con 0x7f9c98068710 2023-12-17T12:03:02.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.310+0000 7f9c8e7fc700 1 -- 172.21.15.192:0/3674488231 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c94022b30 con 0x7f9c98068710 2023-12-17T12:03:02.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.311+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9819f130 con 0x7f9c98068710 2023-12-17T12:03:02.315 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.312+0000 7f9c8e7fc700 1 -- 172.21.15.192:0/3674488231 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f9c9400b960 con 0x7f9c98068710 2023-12-17T12:03:02.315 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.312+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c98100410 con 0x7f9c98068710 2023-12-17T12:03:02.315 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.312+0000 7f9c8e7fc700 1 --2- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c840388c0 0x7f9c8403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:02.315 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.312+0000 7f9c8e7fc700 1 -- 172.21.15.192:0/3674488231 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9c9404cb30 con 0x7f9c98068710 2023-12-17T12:03:02.316 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.313+0000 7f9c9c9b9700 1 --2- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c840388c0 0x7f9c8403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:02.316 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.313+0000 7f9c9c9b9700 1 --2- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c840388c0 0x7f9c8403ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9c88006fd0 tx=0x7f9c88006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:02.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.317+0000 7f9c8e7fc700 1 -- 172.21.15.192:0/3674488231 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9c9400f460 con 0x7f9c98068710 2023-12-17T12:03:02.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.668+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9c9810d2c0 con 0x7f9c98068710 2023-12-17T12:03:02.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.668+0000 7f9c8e7fc700 1 -- 172.21.15.192:0/3674488231 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f9c94027030 con 0x7f9c98068710 2023-12-17T12:03:02.671 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T12:03:02.671 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","modified":"2023-12-17T12:01:21.690077Z","created":"2023-12-17T12:01:21.690077Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-17T12:03:02.673 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c840388c0 msgr2=0x7f9c8403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:02.673 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 --2- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c840388c0 0x7f9c8403ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9c88006fd0 tx=0x7f9c88006e40 comp rx=0 tx=0).stop 2023-12-17T12:03:02.673 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 msgr2=0x7f9c9819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:02.673 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 --2- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 0x7f9c9819e5d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c940098e0 tx=0x7f9c94005710 comp rx=0 tx=0).stop 2023-12-17T12:03:02.673 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 shutdown_connections 2023-12-17T12:03:02.674 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 --2- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c840388c0 0x7f9c8403ad80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:02.674 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 --2- 172.21.15.192:0/3674488231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c98068710 0x7f9c9819e5d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:02.674 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 >> 172.21.15.192:0/3674488231 conn(0x7f9c98075930 msgr2=0x7f9c980fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:02.674 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 shutdown_connections 2023-12-17T12:03:02.674 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:02.671+0000 7f9c9f41e700 1 -- 172.21.15.192:0/3674488231 wait complete. 2023-12-17T12:03:02.674 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2023-12-17T12:03:02.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:02 smithi027 ceph-mon[36871]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:02.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:02 smithi027 ceph-mon[36871]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:03:02.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:02 smithi027 ceph-mon[36871]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-17T12:03:02.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:02.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:02.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:02.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:02.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:03.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:03 smithi027 ceph-mon[36871]: Deploying daemon mon.c on smithi192 2023-12-17T12:03:03.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:03 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/3674488231' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:04.373 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-17T12:03:04.374 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-17T12:03:04.857 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:03:04.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:04 smithi027 ceph-mon[36871]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-12-17T12:03:06.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.error_if_exists: 0 2023-12-17T12:03:06.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.create_if_missing: 0 2023-12-17T12:03:06.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.317+0000 7fd6a97ad700 1 -- 172.21.15.192:0/3441053539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 msgr2=0x7fd6a4101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:06.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.317+0000 7fd6a97ad700 1 --2- 172.21.15.192:0/3441053539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 0x7fd6a4101650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd694009910 tx=0x7fd694009c20 comp rx=0 tx=0).stop 2023-12-17T12:03:06.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.317+0000 7fd6a97ad700 1 -- 172.21.15.192:0/3441053539 shutdown_connections 2023-12-17T12:03:06.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.317+0000 7fd6a97ad700 1 --2- 172.21.15.192:0/3441053539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 0x7fd6a4101650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:06.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.317+0000 7fd6a97ad700 1 -- 172.21.15.192:0/3441053539 >> 172.21.15.192:0/3441053539 conn(0x7fd6a40fcb20 msgr2=0x7fd6a40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:06.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.317+0000 7fd6a97ad700 1 -- 172.21.15.192:0/3441053539 shutdown_connections 2023-12-17T12:03:06.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.317+0000 7fd6a97ad700 1 -- 172.21.15.192:0/3441053539 wait complete. 2023-12-17T12:03:06.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.318+0000 7fd6a97ad700 1 Processor -- start 2023-12-17T12:03:06.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.319+0000 7fd6a97ad700 1 -- start start 2023-12-17T12:03:06.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.319+0000 7fd6a97ad700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 0x7fd6a41a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:06.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.319+0000 7fd6a97ad700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a41a2f50 con 0x7fd6a4101270 2023-12-17T12:03:06.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.319+0000 7fd6a2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 0x7fd6a41a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:06.322 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.319+0000 7fd6a2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 0x7fd6a41a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.192:33004/0 (socket says 172.21.15.192:33004) 2023-12-17T12:03:06.322 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.319+0000 7fd6a2ffd700 1 -- 172.21.15.192:0/4255478859 learned_addr learned my addr 172.21.15.192:0/4255478859 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:03:06.322 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.320+0000 7fd6a2ffd700 1 -- 172.21.15.192:0/4255478859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940095f0 con 0x7fd6a4101270 2023-12-17T12:03:06.322 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.320+0000 7fd6a2ffd700 1 --2- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 0x7fd6a41a2a10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6940098e0 tx=0x7fd694005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:06.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.320+0000 7fd68bfff700 1 -- 172.21.15.192:0/4255478859 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd69401d070 con 0x7fd6a4101270 2023-12-17T12:03:06.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.321+0000 7fd68bfff700 1 -- 172.21.15.192:0/4255478859 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd694009e60 con 0x7fd6a4101270 2023-12-17T12:03:06.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.321+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a41a3150 con 0x7fd6a4101270 2023-12-17T12:03:06.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.321+0000 7fd68bfff700 1 -- 172.21.15.192:0/4255478859 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd694022b30 con 0x7fd6a4101270 2023-12-17T12:03:06.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.321+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a41a3570 con 0x7fd6a4101270 2023-12-17T12:03:06.324 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.322+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a4102750 con 0x7fd6a4101270 2023-12-17T12:03:06.327 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.325+0000 7fd68bfff700 1 -- 172.21.15.192:0/4255478859 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fd694022470 con 0x7fd6a4101270 2023-12-17T12:03:06.327 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.326+0000 7fd68bfff700 1 --2- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd68c0388c0 0x7fd68c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:06.327 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.326+0000 7fd68bfff700 1 -- 172.21.15.192:0/4255478859 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd69404d210 con 0x7fd6a4101270 2023-12-17T12:03:06.328 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.326+0000 7fd6a27fc700 1 --2- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd68c0388c0 0x7fd68c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:06.328 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.327+0000 7fd6a27fc700 1 --2- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd68c0388c0 0x7fd68c03ad80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd698006fd0 tx=0x7fd698006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:06.334 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.332+0000 7fd68bfff700 1 -- 172.21.15.192:0/4255478859 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd69402a930 con 0x7fd6a4101270 2023-12-17T12:03:06.363 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.361+0000 7fd68bfff700 1 -- 172.21.15.192:0/4255478859 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd69402a930 con 0x7fd6a4101270 2023-12-17T12:03:06.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.paranoid_checks: 1 2023-12-17T12:03:06.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-17T12:03:06.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-17T12:03:06.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-17T12:03:06.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.env: 0x5579f52f5a60 2023-12-17T12:03:06.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.fs: PosixFileSystem 2023-12-17T12:03:06.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.info_log: 0x5579f7341780 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_file_opening_threads: 16 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.statistics: (nil) 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.use_fsync: 0 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_log_file_size: 0 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.keep_log_file_num: 1000 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.recycle_log_file_num: 0 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.allow_fallocate: 1 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.allow_mmap_reads: 0 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.allow_mmap_writes: 0 2023-12-17T12:03:06.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.use_direct_reads: 0 2023-12-17T12:03:06.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-17T12:03:06.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.create_missing_column_families: 0 2023-12-17T12:03:06.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.db_log_dir: 2023-12-17T12:03:06.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.wal_dir: 2023-12-17T12:03:06.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-17T12:03:06.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.advise_random_on_open: 1 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.db_write_buffer_size: 0 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.write_buffer_manager: 0x5579f72f7360 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-17T12:03:06.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.rate_limiter: (nil) 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.wal_recovery_mode: 2 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.enable_thread_tracking: 0 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.enable_pipelined_write: 0 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.unordered_write: 0 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.row_cache: None 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.wal_filter: None 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-17T12:03:06.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.allow_ingest_behind: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.two_write_queues: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.manual_wal_flush: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.wal_compression: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.atomic_flush: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.log_readahead_size: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.best_efforts_recovery: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.allow_data_in_errors: 0 2023-12-17T12:03:06.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.db_host_id: __hostname__ 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_background_jobs: 2 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_background_compactions: -1 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_subcompactions: 1 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_total_wal_size: 0 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_open_files: -1 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bytes_per_sync: 0 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-17T12:03:06.568 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_readahead_size: 0 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_background_flushes: -1 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Compression algorithms supported: 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: kZSTD supported: 0 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: kXpressCompression supported: 0 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: kLZ4HCCompression supported: 1 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: kZlibCompression supported: 1 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: kSnappyCompression supported: 1 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: kLZ4Compression supported: 1 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: kBZip2Compression supported: 0 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-17T12:03:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.merge_operator: 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_filter: None 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_filter_factory: None 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.sst_partitioner_factory: None 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5579f73418a0) 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks: 1 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_top_level_index_and_filter: 1 2023-12-17T12:03:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_type: 0 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_index_type: 0 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_shortening: 1 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: checksum: 4 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: no_block_cache: 0 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache: 0x5579f7397090 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_name: BinnedLRUCache 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_options: 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: capacity : 536870912 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_shard_bits : 4 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: strict_capacity_limit : 0 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: high_pri_pool_ratio: 0.000 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_compressed: (nil) 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: persistent_cache: (nil) 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size: 4096 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size_deviation: 10 2023-12-17T12:03:06.571 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_restart_interval: 16 2023-12-17T12:03:06.572 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_block_restart_interval: 1 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: metadata_block_size: 4096 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: partition_filters: 0 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: use_delta_encoding: 1 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: filter_policy: bloomfilter 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: whole_key_filtering: 1 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: verify_compression: 0 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: read_amp_bytes_per_bit: 0 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: format_version: 5 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_index_compression: 1 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_align: 0 2023-12-17T12:03:06.573 INFO:journalctl@ceph.mon.c.smithi192.stdout: max_auto_readahead_size: 262144 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout: prepopulate_block_cache: 0 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout: initial_auto_readahead_size: 8192 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.write_buffer_size: 33554432 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_write_buffer_number: 2 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression: NoCompression 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression: Disabled 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.prefix_extractor: nullptr 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.num_levels: 7 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-17T12:03:06.574 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.level: 32767 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.strategy: 0 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.enabled: false 2023-12-17T12:03:06.575 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.target_file_size_base: 67108864 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-17T12:03:06.576 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.arena_block_size: 1048576 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.disable_auto_compactions: 0 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-17T12:03:06.577 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.table_properties_collectors: 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.inplace_update_support: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.bloom_locality: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.max_successive_merges: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.paranoid_file_checks: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.force_consistency_checks: 1 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.report_bg_io_stats: 0 2023-12-17T12:03:06.578 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.ttl: 2592000 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.enable_blob_files: false 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.min_blob_size: 0 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.blob_file_size: 268435456 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.blob_file_starting_level: 0 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-17T12:03:06.579 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4e7184d7-ec67-4f81-82d3-8d7edc2cc8d1 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814586310689, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814586311771, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702814586, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4e7184d7-ec67-4f81-82d3-8d7edc2cc8d1", "db_session_id": "ZU7M1VGGJ00MZ3CRP032", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814586311980, "job": 1, "event": "recovery_finished"} 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5579f7432000 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: DB pointer 0x5579f741e000 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** DB Stats ** 2023-12-17T12:03:06.580 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:03:06.581 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-17T12:03:06.582 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache BinnedLRUCache@0x5579f7397090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: using public_addr v2:172.21.15.192:0/0 -> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: starting mon.c rank -1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(???) e0 preinit fsid 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 bash[41836]: e573025f2abaef9109ffbd413d4e6c2ea3ed8c93c2300d396ee058167e88cd7e 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 systemd[1]: Started Ceph mon.c for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:03:06.583 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).mds e1 new map 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).mds e1 print_map 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout: e1 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout: legacy client fscid: -1 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout: No filesystems configured 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:03:06.584 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi141", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Updating smithi141:/etc/ceph/ceph.conf 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Added host smithi192 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Updating smithi141:/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:03:06.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/675045840' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/675045840' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.586 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Updating smithi192:/etc/ceph/ceph.conf 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c;count:3 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:03:06.587 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: Deploying daemon mon.c on smithi192 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/3674488231' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:06.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:06 smithi192 ceph-mon[41899]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-17T12:03:06.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:06.621+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd6a4062c00 con 0x7fd6a4101270 2023-12-17T12:03:11.378 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.376+0000 7fd68bfff700 1 -- 172.21.15.192:0/4255478859 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fd694027070 con 0x7fd6a4101270 2023-12-17T12:03:11.378 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T12:03:11.379 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","modified":"2023-12-17T12:03:06.361324Z","created":"2023-12-17T12:01:21.690077Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-17T12:03:11.381 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.379+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd68c0388c0 msgr2=0x7fd68c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.379+0000 7fd6a97ad700 1 --2- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd68c0388c0 0x7fd68c03ad80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd698006fd0 tx=0x7fd698006e40 comp rx=0 tx=0).stop 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.379+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 msgr2=0x7fd6a41a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.379+0000 7fd6a97ad700 1 --2- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 0x7fd6a41a2a10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6940098e0 tx=0x7fd694005710 comp rx=0 tx=0).stop 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.379+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 shutdown_connections 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.379+0000 7fd6a97ad700 1 --2- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd68c0388c0 0x7fd68c03ad80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.379+0000 7fd6a97ad700 1 --2- 172.21.15.192:0/4255478859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6a4101270 0x7fd6a41a2a10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.379+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 >> 172.21.15.192:0/4255478859 conn(0x7fd6a40fcb20 msgr2=0x7fd6a4073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.380+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 shutdown_connections 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:11.380+0000 7fd6a97ad700 1 -- 172.21.15.192:0/4255478859 wait complete. 2023-12-17T12:03:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2023-12-17T12:03:11.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: mon.a calling monitor election 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/4255478859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: mon.c calling monitor election 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: monmap e2: 2 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:11.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: fsmap 2023-12-17T12:03:11.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:11.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: mgrmap e13: x(active, since 48s) 2023-12-17T12:03:11.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: overall HEALTH_OK 2023-12-17T12:03:11.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:11.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:11.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:11.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:11.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:11.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: mon.a calling monitor election 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/4255478859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: mon.c calling monitor election 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: monmap e2: 2 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: fsmap 2023-12-17T12:03:11.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:11.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: mgrmap e13: x(active, since 48s) 2023-12-17T12:03:11.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: overall HEALTH_OK 2023-12-17T12:03:11.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:11.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:11.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:11.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:11.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:12.799 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-17T12:03:12.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-17T12:03:12.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:12 smithi192 ceph-mon[41899]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:12.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:12 smithi192 ceph-mon[41899]: Deploying daemon mon.b on smithi141 2023-12-17T12:03:12.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:12.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:12 smithi027 ceph-mon[36871]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:12.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:12 smithi027 ceph-mon[36871]: Deploying daemon mon.b on smithi141 2023-12-17T12:03:12.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:13.205 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:03:13.663 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:03:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:03:13.360+0000 7fcedb338700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-17T12:03:14.331 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.328+0000 7f91f1956700 1 -- 172.21.15.192:0/785257148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec101270 msgr2=0x7f91ec101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:14.331 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.328+0000 7f91f1956700 1 --2- 172.21.15.192:0/785257148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec101270 0x7f91ec101650 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009910 tx=0x7f91dc009c20 comp rx=0 tx=0).stop 2023-12-17T12:03:14.331 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.328+0000 7f91f1956700 1 -- 172.21.15.192:0/785257148 shutdown_connections 2023-12-17T12:03:14.331 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.328+0000 7f91f1956700 1 --2- 172.21.15.192:0/785257148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec101270 0x7f91ec101650 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:14.331 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.328+0000 7f91f1956700 1 -- 172.21.15.192:0/785257148 >> 172.21.15.192:0/785257148 conn(0x7f91ec0fcb20 msgr2=0x7f91ec0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:14.331 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.329+0000 7f91f1956700 1 -- 172.21.15.192:0/785257148 shutdown_connections 2023-12-17T12:03:14.331 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.329+0000 7f91f1956700 1 -- 172.21.15.192:0/785257148 wait complete. 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.329+0000 7f91f1956700 1 Processor -- start 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.330+0000 7f91f1956700 1 -- start start 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.330+0000 7f91f1956700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 0x7f91ec19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.330+0000 7f91f1956700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec19ed90 0x7f91ec198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.330+0000 7f91f1956700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19f210 con 0x7f91ec19ed90 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.330+0000 7f91f1956700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19f380 con 0x7f91ec101270 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91eaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 0x7f91ec19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91eaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 0x7f91ec19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:49754/0 (socket says 172.21.15.192:49754) 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91eaffd700 1 -- 172.21.15.192:0/1164937478 learned_addr learned my addr 172.21.15.192:0/1164937478 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91eaffd700 1 -- 172.21.15.192:0/1164937478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 msgr2=0x7f91ec19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-17T12:03:14.333 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91eaffd700 1 -- 172.21.15.192:0/1164937478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 msgr2=0x7f91ec19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-17T12:03:14.334 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91eaffd700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 0x7f91ec19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-17T12:03:14.334 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91ea7fc700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec19ed90 0x7f91ec198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:14.334 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91eaffd700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 0x7f91ec19e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91ea7fc700 1 -- 172.21.15.192:0/1164937478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 msgr2=0x7f91ec19e850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91ea7fc700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 0x7f91ec19e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.331+0000 7f91ea7fc700 1 -- 172.21.15.192:0/1164937478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc0095f0 con 0x7f91ec19ed90 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.332+0000 7f91ea7fc700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec19ed90 0x7f91ec198990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f91e000b4d0 tx=0x7f91e000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.332+0000 7f91f0954700 1 -- 172.21.15.192:0/1164937478 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f91e0011630 con 0x7f91ec19ed90 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.332+0000 7f91f0954700 1 -- 172.21.15.192:0/1164937478 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91e0011c20 con 0x7f91ec19ed90 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.332+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec198ff0 con 0x7f91ec19ed90 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.332+0000 7f91f0954700 1 -- 172.21.15.192:0/1164937478 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f91e0010c60 con 0x7f91ec19ed90 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.332+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec1994f0 con 0x7f91ec19ed90 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.333+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ec1997c0 con 0x7f91ec19ed90 2023-12-17T12:03:14.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.333+0000 7f91f0954700 1 -- 172.21.15.192:0/1164937478 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f91e0011790 con 0x7f91ec19ed90 2023-12-17T12:03:14.336 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.333+0000 7f91f0954700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f91d40386a0 0x7f91d403ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:14.336 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.334+0000 7f91f0954700 1 -- 172.21.15.192:0/1164937478 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91e004af60 con 0x7f91ec19ed90 2023-12-17T12:03:14.336 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.334+0000 7f91eaffd700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f91d40386a0 0x7f91d403ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:14.336 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.335+0000 7f91eaffd700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f91d40386a0 0x7f91d403ab60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00b5c0 tx=0x7f91dc009ec0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:14.341 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.339+0000 7f91f0954700 1 -- 172.21.15.192:0/1164937478 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f91e00172c0 con 0x7f91ec19ed90 2023-12-17T12:03:14.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 systemd[1]: Starting Ceph mon.b for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:03:14.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.614+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f91ec102750 con 0x7f91ec19ed90 2023-12-17T12:03:14.617 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.615+0000 7f91f0954700 1 -- 172.21.15.192:0/1164937478 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f91e00172c0 con 0x7f91ec19ed90 2023-12-17T12:03:14.617 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T12:03:14.617 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","modified":"2023-12-17T12:03:06.361324Z","created":"2023-12-17T12:01:21.690077Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-17T12:03:14.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f91d40386a0 msgr2=0x7f91d403ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:14.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f91d40386a0 0x7f91d403ab60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00b5c0 tx=0x7f91dc009ec0 comp rx=0 tx=0).stop 2023-12-17T12:03:14.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec19ed90 msgr2=0x7f91ec198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:14.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec19ed90 0x7f91ec198990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f91e000b4d0 tx=0x7f91e000b890 comp rx=0 tx=0).stop 2023-12-17T12:03:14.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 shutdown_connections 2023-12-17T12:03:14.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f91d40386a0 0x7f91d403ab60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:14.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec101270 0x7f91ec19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:14.620 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 --2- 172.21.15.192:0/1164937478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ec19ed90 0x7f91ec198990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:14.620 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 >> 172.21.15.192:0/1164937478 conn(0x7f91ec0fcb20 msgr2=0x7f91ec0738a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:14.620 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 shutdown_connections 2023-12-17T12:03:14.620 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:14.617+0000 7f91f1956700 1 -- 172.21.15.192:0/1164937478 wait complete. 2023-12-17T12:03:14.620 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2023-12-17T12:03:14.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:14 smithi192 ceph-mon[41899]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:14.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:14 smithi027 ceph-mon[36871]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 bash[41187]: 93d686ce02bfc456e3a916180c792aaed38ef767112b9cbae11d712f3f2a0719 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 systemd[1]: Started Ceph mon.b for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: set uid:gid to 167:167 (ceph:ceph) 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev), process ceph-mon, pid 7 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: pidfile_write: ignore empty --pid-file 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: load: jerasure load: lrc 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: RocksDB version: 7.9.2 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Git sha 0 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Compile date 2023-12-11 11:29:13 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: DB SUMMARY 2023-12-17T12:03:15.267 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: DB Session ID: K2QY9S17VA8CTV8TK21Q 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: CURRENT file: CURRENT 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: IDENTITY file: IDENTITY 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.error_if_exists: 0 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.create_if_missing: 0 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.paranoid_checks: 1 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.env: 0x56076305da60 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.fs: PosixFileSystem 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.info_log: 0x560764b49780 2023-12-17T12:03:15.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_file_opening_threads: 16 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.statistics: (nil) 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.use_fsync: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_log_file_size: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.keep_log_file_num: 1000 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.recycle_log_file_num: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.allow_fallocate: 1 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.allow_mmap_reads: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.allow_mmap_writes: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.use_direct_reads: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.create_missing_column_families: 0 2023-12-17T12:03:15.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.db_log_dir: 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.wal_dir: 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.advise_random_on_open: 1 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.db_write_buffer_size: 0 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.write_buffer_manager: 0x560764aff360 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.rate_limiter: (nil) 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-17T12:03:15.270 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.wal_recovery_mode: 2 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.enable_thread_tracking: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.enable_pipelined_write: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.unordered_write: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.row_cache: None 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.wal_filter: None 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.allow_ingest_behind: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.two_write_queues: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.manual_wal_flush: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.wal_compression: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.atomic_flush: 0 2023-12-17T12:03:15.271 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.log_readahead_size: 0 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.best_efforts_recovery: 0 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.allow_data_in_errors: 0 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.db_host_id: __hostname__ 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_background_jobs: 2 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_background_compactions: -1 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_subcompactions: 1 2023-12-17T12:03:15.272 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_total_wal_size: 0 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_open_files: -1 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bytes_per_sync: 0 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_readahead_size: 0 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_background_flushes: -1 2023-12-17T12:03:15.273 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Compression algorithms supported: 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: kZSTD supported: 0 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: kXpressCompression supported: 0 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: kLZ4HCCompression supported: 1 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: kZlibCompression supported: 1 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: kSnappyCompression supported: 1 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: kLZ4Compression supported: 1 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: kBZip2Compression supported: 0 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.merge_operator: 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_filter: None 2023-12-17T12:03:15.274 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_filter_factory: None 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.sst_partitioner_factory: None 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560764b498a0) 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: cache_index_and_filter_blocks: 1 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: pin_top_level_index_and_filter: 1 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: index_type: 0 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: data_block_index_type: 0 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: index_shortening: 1 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: checksum: 4 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: no_block_cache: 0 2023-12-17T12:03:15.275 INFO:journalctl@ceph.mon.b.smithi141.stdout: block_cache: 0x560764b9f090 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: block_cache_name: BinnedLRUCache 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: block_cache_options: 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: capacity : 536870912 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: num_shard_bits : 4 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: strict_capacity_limit : 0 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: high_pri_pool_ratio: 0.000 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: block_cache_compressed: (nil) 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: persistent_cache: (nil) 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: block_size: 4096 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: block_size_deviation: 10 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: block_restart_interval: 16 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: index_block_restart_interval: 1 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: metadata_block_size: 4096 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: partition_filters: 0 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: use_delta_encoding: 1 2023-12-17T12:03:15.276 INFO:journalctl@ceph.mon.b.smithi141.stdout: filter_policy: bloomfilter 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: whole_key_filtering: 1 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: verify_compression: 0 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: read_amp_bytes_per_bit: 0 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: format_version: 5 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: enable_index_compression: 1 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: block_align: 0 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: max_auto_readahead_size: 262144 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: prepopulate_block_cache: 0 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: initial_auto_readahead_size: 8192 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout: num_file_reads_for_auto_readahead: 2 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.write_buffer_size: 33554432 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_write_buffer_number: 2 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression: NoCompression 2023-12-17T12:03:15.277 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression: Disabled 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.prefix_extractor: nullptr 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.num_levels: 7 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-17T12:03:15.278 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.level: 32767 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.strategy: 0 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.enabled: false 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.target_file_size_base: 67108864 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-17T12:03:15.279 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.arena_block_size: 1048576 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.disable_auto_compactions: 0 2023-12-17T12:03:15.280 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.table_properties_collectors: 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.inplace_update_support: 0 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-17T12:03:15.281 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.bloom_locality: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.max_successive_merges: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.paranoid_file_checks: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.force_consistency_checks: 1 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.report_bg_io_stats: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.ttl: 2592000 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.enable_blob_files: false 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.min_blob_size: 0 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.blob_file_size: 268435456 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-17T12:03:15.282 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.blob_file_starting_level: 0 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2854c047-d976-4e10-afec-7aeb4f64224a 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814594965776, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814594967114, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702814594, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2854c047-d976-4e10-afec-7aeb4f64224a", "db_session_id": "K2QY9S17VA8CTV8TK21Q", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702814594967391, "job": 1, "event": "recovery_finished"} 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-17T12:03:15.283 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560764c3a000 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: DB pointer 0x560764c26000 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: ** DB Stats ** 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: ** Compaction Stats [default] ** 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-17T12:03:15.284 INFO:journalctl@ceph.mon.b.smithi141.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: ** Compaction Stats [default] ** 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-17T12:03:15.285 INFO:journalctl@ceph.mon.b.smithi141.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: Cumulative compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: Interval compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: Block cache BinnedLRUCache@0x560764b9f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: using public_addr v2:172.21.15.141:0/0 -> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: starting mon.b rank -1 at public addrs [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] at bind addrs [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:14 smithi141 ceph-mon[41225]: mon.b@-1(???) e0 preinit fsid 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:03:15.286 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).mds e1 new map 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).mds e1 print_map 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout: e1 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout: legacy client fscid: -1 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout: 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout: No filesystems configured 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-17T12:03:15.287 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi141", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:15.288 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Updating smithi141:/etc/ceph/ceph.conf 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Added host smithi192 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Updating smithi141:/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.289 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/675045840' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/675045840' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Updating smithi192:/etc/ceph/ceph.conf 2023-12-17T12:03:15.290 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c;count:3 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:15.291 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Deploying daemon mon.c on smithi192 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/3674488231' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.a calling monitor election 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/4255478859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.c calling monitor election 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:15.292 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: monmap e2: 2 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: fsmap 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mgrmap e13: x(active, since 48s) 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: overall HEALTH_OK 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.293 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.294 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:15.294 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:15.294 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:15.294 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.294 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: Deploying daemon mon.b on smithi141 2023-12-17T12:03:15.294 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:15.294 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:15.294 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:15 smithi141 ceph-mon[41225]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-17T12:03:16.019 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-17T12:03:16.019 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-17T12:03:16.423 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: mon.a calling monitor election 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: mon.c calling monitor election 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: monmap e3: 3 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],b=[v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: fsmap 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: mgrmap e13: x(active, since 57s) 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: mon.b (rank 2) addr [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] is down (out of quorum) 2023-12-17T12:03:20.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:20.415 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:20.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: mon.a calling monitor election 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: mon.c calling monitor election 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-17T12:03:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: monmap e3: 3 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],b=[v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: fsmap 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: mgrmap e13: x(active, since 57s) 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: mon.b (rank 2) addr [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] is down (out of quorum) 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:20.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:20.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.613+0000 7fbf88b21700 1 -- 172.21.15.192:0/630886072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 msgr2=0x7fbf6c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:20.617 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.613+0000 7fbf88b21700 1 --2- 172.21.15.192:0/630886072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 0x7fbf6c005a50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf84100f40 tx=0x7fbf7400feb0 comp rx=0 tx=0).stop 2023-12-17T12:03:20.617 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.613+0000 7fbf88b21700 1 -- 172.21.15.192:0/630886072 shutdown_connections 2023-12-17T12:03:20.617 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.613+0000 7fbf88b21700 1 --2- 172.21.15.192:0/630886072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 0x7fbf6c005a50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:20.617 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.613+0000 7fbf88b21700 1 -- 172.21.15.192:0/630886072 >> 172.21.15.192:0/630886072 conn(0x7fbf840fcb20 msgr2=0x7fbf840fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:20.617 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.614+0000 7fbf88b21700 1 -- 172.21.15.192:0/630886072 shutdown_connections 2023-12-17T12:03:20.617 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.614+0000 7fbf88b21700 1 -- 172.21.15.192:0/630886072 wait complete. 2023-12-17T12:03:20.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.615+0000 7fbf88b21700 1 Processor -- start 2023-12-17T12:03:20.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.615+0000 7fbf88b21700 1 -- start start 2023-12-17T12:03:20.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.616+0000 7fbf88b21700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 0x7fbf8419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:20.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.616+0000 7fbf88b21700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 0x7fbf84196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:20.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.616+0000 7fbf88b21700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf84196c60 0x7fbf841970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:20.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.616+0000 7fbf88b21700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8419d0d0 con 0x7fbf6c0035c0 2023-12-17T12:03:20.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.616+0000 7fbf88b21700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf841a33b0 con 0x7fbf8419cad0 2023-12-17T12:03:20.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.616+0000 7fbf88b21700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf841a34f0 con 0x7fbf84196c60 2023-12-17T12:03:20.621 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.616+0000 7fbf8259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 0x7fbf8419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:20.621 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf8259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 0x7fbf8419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.192:53324/0 (socket says 172.21.15.192:53324) 2023-12-17T12:03:20.621 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf8259c700 1 -- 172.21.15.192:0/2557026588 learned_addr learned my addr 172.21.15.192:0/2557026588 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:03:20.621 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf81d9b700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 0x7fbf84196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:20.621 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf82d9d700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf84196c60 0x7fbf841970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:20.621 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf81d9b700 1 -- 172.21.15.192:0/2557026588 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 msgr2=0x7fbf84196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-12-17T12:03:20.621 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf81d9b700 1 -- 172.21.15.192:0/2557026588 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 msgr2=0x7fbf84196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf81d9b700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 0x7fbf84196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf81d9b700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 0x7fbf84196720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf8259c700 1 -- 172.21.15.192:0/2557026588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf84196c60 msgr2=0x7fbf841970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf8259c700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf84196c60 0x7fbf841970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf8259c700 1 -- 172.21.15.192:0/2557026588 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 msgr2=0x7fbf84196720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf8259c700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 0x7fbf84196720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf8259c700 1 -- 172.21.15.192:0/2557026588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7400f900 con 0x7fbf6c0035c0 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.617+0000 7fbf8259c700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 0x7fbf8419c590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400f8d0 tx=0x7fbf74013960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.618+0000 7fbf737fe700 1 -- 172.21.15.192:0/2557026588 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74004900 con 0x7fbf6c0035c0 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.618+0000 7fbf737fe700 1 -- 172.21.15.192:0/2557026588 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf74004030 con 0x7fbf6c0035c0 2023-12-17T12:03:20.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.618+0000 7fbf737fe700 1 -- 172.21.15.192:0/2557026588 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf740039d0 con 0x7fbf6c0035c0 2023-12-17T12:03:20.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.618+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf841a3770 con 0x7fbf6c0035c0 2023-12-17T12:03:20.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.618+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf841a3c70 con 0x7fbf6c0035c0 2023-12-17T12:03:20.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.619+0000 7fbf737fe700 1 -- 172.21.15.192:0/2557026588 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fbf740041a0 con 0x7fbf6c0035c0 2023-12-17T12:03:20.624 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.619+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf84102750 con 0x7fbf6c0035c0 2023-12-17T12:03:20.624 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.619+0000 7fbf737fe700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbf64038d40 0x7fbf6403b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:20.624 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.619+0000 7fbf737fe700 1 -- 172.21.15.192:0/2557026588 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbf74000e40 con 0x7fbf6c0035c0 2023-12-17T12:03:20.624 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.620+0000 7fbf81d9b700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbf64038d40 0x7fbf6403b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:20.624 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.620+0000 7fbf81d9b700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbf64038d40 0x7fbf6403b200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009850 tx=0x7fbf78008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:20.627 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.625+0000 7fbf737fe700 1 -- 172.21.15.192:0/2557026588 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbf7402dc00 con 0x7fbf6c0035c0 2023-12-17T12:03:20.925 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.922+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbf84108c40 con 0x7fbf6c0035c0 2023-12-17T12:03:20.925 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.922+0000 7fbf737fe700 1 -- 172.21.15.192:0/2557026588 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7fbf7401b3f0 con 0x7fbf6c0035c0 2023-12-17T12:03:20.925 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-17T12:03:20.925 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","modified":"2023-12-17T12:03:15.013112Z","created":"2023-12-17T12:01:21.690077Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:3300","nonce":0},{"type":"v1","addr":"172.21.15.141:6789","nonce":0}]},"addr":"172.21.15.141:6789/0","public_addr":"172.21.15.141:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-17T12:03:20.928 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbf64038d40 msgr2=0x7fbf6403b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:20.928 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbf64038d40 0x7fbf6403b200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009850 tx=0x7fbf78008040 comp rx=0 tx=0).stop 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 msgr2=0x7fbf8419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 0x7fbf8419c590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400f8d0 tx=0x7fbf74013960 comp rx=0 tx=0).stop 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 shutdown_connections 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbf64038d40 0x7fbf6403b200 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf84196c60 0x7fbf841970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6c0035c0 0x7fbf8419c590 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 --2- 172.21.15.192:0/2557026588 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbf8419cad0 0x7fbf84196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 >> 172.21.15.192:0/2557026588 conn(0x7fbf840fcb20 msgr2=0x7fbf84104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.925+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 shutdown_connections 2023-12-17T12:03:20.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:20.926+0000 7fbf88b21700 1 -- 172.21.15.192:0/2557026588 wait complete. 2023-12-17T12:03:20.930 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2023-12-17T12:03:21.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:21.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:21.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:21.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:21 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/2557026588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:21.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:21.330 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-17T12:03:21.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph config generate-minimal-conf 2023-12-17T12:03:21.401 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:21.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:21.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:21.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:21 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/2557026588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:21.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:21.765 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:03:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:03:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.a calling monitor election 2023-12-17T12:03:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.c calling monitor election 2023-12-17T12:03:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:03:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: monmap e3: 3 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],b=[v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: fsmap 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mgrmap e13: x(active, since 57s) 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-17T12:03:22.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.b (rank 2) addr [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] is down (out of quorum) 2023-12-17T12:03:22.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:22.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:22.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:22.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:22.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:22.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/2557026588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-17T12:03:22.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: mon.b calling monitor election 2023-12-17T12:03:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: Updating smithi027:/etc/ceph/ceph.conf 2023-12-17T12:03:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: Updating smithi141:/etc/ceph/ceph.conf 2023-12-17T12:03:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: Updating smithi192:/etc/ceph/ceph.conf 2023-12-17T12:03:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: mon.b calling monitor election 2023-12-17T12:03:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: mon.a calling monitor election 2023-12-17T12:03:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: mon.c calling monitor election 2023-12-17T12:03:22.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: monmap e3: 3 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],b=[v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: fsmap 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: mgrmap e13: x(active, since 59s) 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: Cluster is now healthy 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: overall HEALTH_OK 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:22.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:22 smithi027 ceph-mon[36871]: Updating smithi027:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.b calling monitor election 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Updating smithi027:/etc/ceph/ceph.conf 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Updating smithi141:/etc/ceph/ceph.conf 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Updating smithi192:/etc/ceph/ceph.conf 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.b calling monitor election 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.a calling monitor election 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.c calling monitor election 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: monmap e3: 3 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],b=[v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: fsmap 2023-12-17T12:03:23.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:23.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: mgrmap e13: x(active, since 59s) 2023-12-17T12:03:23.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-17T12:03:23.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Cluster is now healthy 2023-12-17T12:03:23.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: overall HEALTH_OK 2023-12-17T12:03:23.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:23.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:23.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:22 smithi141 ceph-mon[41225]: Updating smithi027:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:23.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: mon.b calling monitor election 2023-12-17T12:03:23.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: Updating smithi027:/etc/ceph/ceph.conf 2023-12-17T12:03:23.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: Updating smithi141:/etc/ceph/ceph.conf 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: Updating smithi192:/etc/ceph/ceph.conf 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: mon.b calling monitor election 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: mon.a calling monitor election 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: mon.c calling monitor election 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: monmap e3: 3 mons at {a=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],b=[v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: fsmap 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: osdmap e4: 0 total, 0 up, 0 in 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: mgrmap e13: x(active, since 59s) 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: Cluster is now healthy 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: overall HEALTH_OK 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: Updating smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: Updating smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:23.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:22 smithi192 ceph-mon[41899]: Updating smithi027:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-17T12:03:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.093+0000 7f79a37e6700 1 -- 172.21.15.27:0/1878337551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 msgr2=0x7f799c101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.093+0000 7f79a37e6700 1 --2- 172.21.15.27:0/1878337551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 0x7f799c101520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7998009910 tx=0x7f7998009c20 comp rx=0 tx=0).stop 2023-12-17T12:03:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.094+0000 7f79a37e6700 1 -- 172.21.15.27:0/1878337551 shutdown_connections 2023-12-17T12:03:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.094+0000 7f79a37e6700 1 --2- 172.21.15.27:0/1878337551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 0x7f799c101520 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.094+0000 7f79a37e6700 1 -- 172.21.15.27:0/1878337551 >> 172.21.15.27:0/1878337551 conn(0x7f799c0fc9d0 msgr2=0x7f799c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.094+0000 7f79a37e6700 1 -- 172.21.15.27:0/1878337551 shutdown_connections 2023-12-17T12:03:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.094+0000 7f79a37e6700 1 -- 172.21.15.27:0/1878337551 wait complete. 2023-12-17T12:03:23.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.095+0000 7f79a37e6700 1 Processor -- start 2023-12-17T12:03:23.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.095+0000 7f79a37e6700 1 -- start start 2023-12-17T12:03:23.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.095+0000 7f79a37e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 0x7f799c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:23.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.095+0000 7f79a37e6700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f799c19ed90 0x7f799c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.095+0000 7f79a37e6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f799c198ed0 0x7f799c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.095+0000 7f79a37e6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c19f390 con 0x7f799c101140 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.095+0000 7f79a37e6700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c1a5670 con 0x7f799c19ed90 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.095+0000 7f79a37e6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c1a57b0 con 0x7f799c198ed0 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 0x7f799c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 0x7f799c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52722/0 (socket says 172.21.15.27:52722) 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 -- 172.21.15.27:0/268752007 learned_addr learned my addr 172.21.15.27:0/268752007 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1d83700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f799c198ed0 0x7f799c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 -- 172.21.15.27:0/268752007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f799c198ed0 msgr2=0x7f799c199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f799c198ed0 0x7f799c199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 -- 172.21.15.27:0/268752007 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f799c19ed90 msgr2=0x7f799c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f799c19ed90 0x7f799c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:23.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 -- 172.21.15.27:0/268752007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79980095f0 con 0x7f799c101140 2023-12-17T12:03:23.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a1582700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 0x7f799c19e850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f799800bd20 tx=0x7f799800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:23.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79927fc700 1 -- 172.21.15.27:0/268752007 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799801d070 con 0x7f799c101140 2023-12-17T12:03:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79927fc700 1 -- 172.21.15.27:0/268752007 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7998004db0 con 0x7f799c101140 2023-12-17T12:03:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79927fc700 1 -- 172.21.15.27:0/268752007 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998022c10 con 0x7f799c101140 2023-12-17T12:03:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c1a5a30 con 0x7f799c101140 2023-12-17T12:03:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.097+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c1a5e80 con 0x7f799c101140 2023-12-17T12:03:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.098+0000 7f79927fc700 1 -- 172.21.15.27:0/268752007 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f7998022470 con 0x7f799c101140 2023-12-17T12:03:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.098+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c102620 con 0x7f799c101140 2023-12-17T12:03:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.098+0000 7f79927fc700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7988038d90 0x7f798803b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.099+0000 7f79927fc700 1 -- 172.21.15.27:0/268752007 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f799804c950 con 0x7f799c101140 2023-12-17T12:03:23.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.099+0000 7f79a0d81700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7988038d90 0x7f798803b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:23.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.099+0000 7f79a0d81700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7988038d90 0x7f798803b250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f798c0098e0 tx=0x7f798c006dd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:23.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.104+0000 7f79927fc700 1 -- 172.21.15.27:0/268752007 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f799800e5c0 con 0x7f799c101140 2023-12-17T12:03:23.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.321+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f799c10b110 con 0x7f799c101140 2023-12-17T12:03:23.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.322+0000 7f79927fc700 1 -- 172.21.15.27:0/268752007 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f7998027020 con 0x7f799c101140 2023-12-17T12:03:23.324 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:03:23.324 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2023-12-17T12:03:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:03:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7988038d90 msgr2=0x7f798803b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7988038d90 0x7f798803b250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f798c0098e0 tx=0x7f798c006dd0 comp rx=0 tx=0).stop 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 msgr2=0x7f799c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 0x7f799c19e850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f799800bd20 tx=0x7f799800be00 comp rx=0 tx=0).stop 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 shutdown_connections 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7988038d90 0x7f798803b250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f799c198ed0 0x7f799c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f799c101140 0x7f799c19e850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:23.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 --2- 172.21.15.27:0/268752007 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f799c19ed90 0x7f799c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:23.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.324+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 >> 172.21.15.27:0/268752007 conn(0x7f799c0fc9d0 msgr2=0x7f799c074880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:23.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.325+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 shutdown_connections 2023-12-17T12:03:23.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:23.325+0000 7f79a37e6700 1 -- 172.21.15.27:0/268752007 wait complete. 2023-12-17T12:03:23.741 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-17T12:03:23.741 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:03:23.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2023-12-17T12:03:23.822 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:03:23.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:03:23.932 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T12:03:23.932 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/etc/ceph/ceph.conf 2023-12-17T12:03:24.011 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T12:03:24.011 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: Reconfiguring mon.a (unknown last config time)... 2023-12-17T12:03:24.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:24.063 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:24.063 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:24.063 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: Reconfiguring daemon mon.a on smithi027 2023-12-17T12:03:24.063 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:23 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/268752007' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: Reconfiguring mon.a (unknown last config time)... 2023-12-17T12:03:24.080 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:24.081 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:24.081 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:24.081 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: Reconfiguring daemon mon.a on smithi027 2023-12-17T12:03:24.081 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:23 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/268752007' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:24.113 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:24.113 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.113 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.113 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.113 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.113 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.113 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: Reconfiguring mon.a (unknown last config time)... 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: Reconfiguring daemon mon.a on smithi027 2023-12-17T12:03:24.114 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:23 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/268752007' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:24.122 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T12:03:24.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2023-12-17T12:03:24.197 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T12:03:24.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-17T12:03:24.306 INFO:tasks.cephadm:Adding mgr.x on smithi027 2023-12-17T12:03:24.306 INFO:tasks.cephadm:Adding mgr.y on smithi141 2023-12-17T12:03:24.306 INFO:tasks.cephadm:Adding mgr.z on smithi192 2023-12-17T12:03:24.306 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch apply mgr '3;smithi027=x;smithi141=y;smithi192=z' 2023-12-17T12:03:24.736 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:03:25.865 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.861+0000 7f1676bdc700 1 -- 172.21.15.192:0/2661933712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1670103290 msgr2=0x7f16701036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:25.865 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.861+0000 7f1676bdc700 1 --2- 172.21.15.192:0/2661933712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1670103290 0x7f16701036b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1664009910 tx=0x7f1664009c20 comp rx=0 tx=0).stop 2023-12-17T12:03:25.866 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.861+0000 7f1676bdc700 1 -- 172.21.15.192:0/2661933712 shutdown_connections 2023-12-17T12:03:25.866 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.861+0000 7f1676bdc700 1 --2- 172.21.15.192:0/2661933712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1670103290 0x7f16701036b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:25.866 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.861+0000 7f1676bdc700 1 -- 172.21.15.192:0/2661933712 >> 172.21.15.192:0/2661933712 conn(0x7f16700fe830 msgr2=0x7f1670100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:25.866 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.862+0000 7f1676bdc700 1 -- 172.21.15.192:0/2661933712 shutdown_connections 2023-12-17T12:03:25.866 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.862+0000 7f1676bdc700 1 -- 172.21.15.192:0/2661933712 wait complete. 2023-12-17T12:03:25.866 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1676bdc700 1 Processor -- start 2023-12-17T12:03:25.866 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1676bdc700 1 -- start start 2023-12-17T12:03:25.867 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1676bdc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1670103290 0x7f1670198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:25.867 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1676bdc700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1670198c90 0x7f167019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:25.867 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1676bdc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1670199110 0x7f16701a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:25.867 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1676bdc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16701a09e0 con 0x7f1670199110 2023-12-17T12:03:25.867 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1676bdc700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16701a0b50 con 0x7f1670198c90 2023-12-17T12:03:25.867 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1676bdc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16701a0d10 con 0x7f1670103290 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1674978700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1670103290 0x7f1670198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1674978700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1670103290 0x7f1670198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:34950/0 (socket says 172.21.15.192:34950) 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.863+0000 7f1674978700 1 -- 172.21.15.192:0/3926337848 learned_addr learned my addr 172.21.15.192:0/3926337848 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f166ffff700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1670198c90 0x7f167019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f1675179700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1670199110 0x7f16701a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f1674978700 1 -- 172.21.15.192:0/3926337848 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1670198c90 msgr2=0x7f167019dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f1674978700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1670198c90 0x7f167019dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f1674978700 1 -- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1670199110 msgr2=0x7f16701a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f1674978700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1670199110 0x7f16701a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f1674978700 1 -- 172.21.15.192:0/3926337848 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640095f0 con 0x7f1670103290 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f166ffff700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1670198c90 0x7f167019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:03:25.868 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.864+0000 7f1674978700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1670103290 0x7f1670198750 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1664000c00 tx=0x7f166400bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:25.869 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.865+0000 7f166dffb700 1 -- 172.21.15.192:0/3926337848 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166401d070 con 0x7f1670103290 2023-12-17T12:03:25.869 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.865+0000 7f166dffb700 1 -- 172.21.15.192:0/3926337848 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f166400bed0 con 0x7f1670103290 2023-12-17T12:03:25.869 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.865+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16701a0f90 con 0x7f1670103290 2023-12-17T12:03:25.869 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.865+0000 7f166dffb700 1 -- 172.21.15.192:0/3926337848 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664022b60 con 0x7f1670103290 2023-12-17T12:03:25.869 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.865+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16701a1420 con 0x7f1670103290 2023-12-17T12:03:25.869 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.866+0000 7f166dffb700 1 -- 172.21.15.192:0/3926337848 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f166400f460 con 0x7f1670103290 2023-12-17T12:03:25.869 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.866+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1670062c70 con 0x7f1670103290 2023-12-17T12:03:25.870 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.867+0000 7f166dffb700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1660038d90 0x7f166003b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:25.870 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.867+0000 7f166dffb700 1 -- 172.21.15.192:0/3926337848 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f166404c960 con 0x7f1670103290 2023-12-17T12:03:25.870 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.867+0000 7f166ffff700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1660038d90 0x7f166003b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:25.870 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.868+0000 7f166ffff700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1660038d90 0x7f166003b250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f165c009ec0 tx=0x7f165c009210 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:25.875 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:25.873+0000 7f166dffb700 1 -- 172.21.15.192:0/3926337848 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1670062c70 con 0x7f1670103290 2023-12-17T12:03:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:25.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:26.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:26.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:26.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:26.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:26.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:26.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:26.110 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.107+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi141=y;smithi192=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f1670062c00 con 0x7f1660038d90 2023-12-17T12:03:26.117 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.114+0000 7f166dffb700 1 -- 172.21.15.192:0/3926337848 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1670062c00 con 0x7f1660038d90 2023-12-17T12:03:26.117 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.117+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1660038d90 msgr2=0x7f166003b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1660038d90 0x7f166003b250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f165c009ec0 tx=0x7f165c009210 comp rx=0 tx=0).stop 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1670103290 msgr2=0x7f1670198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1670103290 0x7f1670198750 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1664000c00 tx=0x7f166400bcf0 comp rx=0 tx=0).stop 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 shutdown_connections 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1660038d90 0x7f166003b250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1670103290 0x7f1670198750 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1670199110 0x7f16701a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 --2- 172.21.15.192:0/3926337848 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1670198c90 0x7f167019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 >> 172.21.15.192:0/3926337848 conn(0x7f16700fe830 msgr2=0x7f1670100180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 shutdown_connections 2023-12-17T12:03:26.121 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:03:26.118+0000 7f1676bdc700 1 -- 172.21.15.192:0/3926337848 wait complete. 2023-12-17T12:03:26.554 DEBUG:teuthology.orchestra.run.smithi141:mgr.y> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.y.service 2023-12-17T12:03:26.559 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.z.service 2023-12-17T12:03:26.563 INFO:tasks.cephadm:Deploying OSDs... 2023-12-17T12:03:26.563 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:03:26.564 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2023-12-17T12:03:26.613 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-17T12:03:26.613 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2023-12-17T12:03:26.633 INFO:journalctl@ceph.mgr.y.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:03:26.635 INFO:journalctl@ceph.mgr.z.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-12-17 12:02:33.967573132 +0000 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-12-17 11:50:45.823316700 +0000 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-12-17 11:50:45.823316700 +0000 2023-12-17T12:03:26.661 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-12-17 11:50:45.794317474 +0000 2023-12-17T12:03:26.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-17T12:03:26.733 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-17T12:03:26.733 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-12-17T12:03:26.733 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000317154 s, 1.6 MB/s 2023-12-17T12:03:26.736 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-17T12:03:26.782 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2023-12-17T12:03:26.793 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: Reconfiguring mon.b (monmap changed)... 2023-12-17T12:03:26.793 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: Reconfiguring daemon mon.b on smithi141 2023-12-17T12:03:26.793 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:26.793 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.794 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.794 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:26.794 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:26.794 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:26.794 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-12-17 12:02:34.036571290 +0000 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-12-17 11:50:46.671294046 +0000 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-12-17 11:50:46.671294046 +0000 2023-12-17T12:03:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-12-17 11:50:46.644294767 +0000 2023-12-17T12:03:26.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-17T12:03:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: Reconfiguring mon.b (monmap changed)... 2023-12-17T12:03:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: Reconfiguring daemon mon.b on smithi141 2023-12-17T12:03:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:26.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:26.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:26.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:26.974 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-17T12:03:26.974 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-12-17T12:03:26.974 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000305988 s, 1.7 MB/s 2023-12-17T12:03:26.977 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: Reconfiguring mon.b (monmap changed)... 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: Reconfiguring daemon mon.b on smithi141 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:27.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:27.064 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-12-17 12:02:34.108569368 +0000 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-12-17 11:50:47.515271499 +0000 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-12-17 11:50:47.515271499 +0000 2023-12-17T12:03:27.111 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-12-17 11:50:47.489272194 +0000 2023-12-17T12:03:27.111 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-17T12:03:27.218 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-17T12:03:27.218 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-12-17T12:03:27.218 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000318304 s, 1.6 MB/s 2023-12-17T12:03:27.221 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-17T12:03:27.268 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2023-12-17T12:03:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-17T12:03:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:27.316 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-12-17T12:03:27.316 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:27.316 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:27.316 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-12-17 12:02:34.181567420 +0000 2023-12-17T12:03:27.316 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-12-17 11:50:48.465246121 +0000 2023-12-17T12:03:27.316 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-12-17 11:50:48.465246121 +0000 2023-12-17T12:03:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-12-17 11:50:48.437246869 +0000 2023-12-17T12:03:27.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-17T12:03:27.387 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-17T12:03:27.387 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-12-17T12:03:27.387 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000286139 s, 1.8 MB/s 2023-12-17T12:03:27.391 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-17T12:03:27.439 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-12-17T12:03:27.439 DEBUG:teuthology.orchestra.run.smithi141:> dd if=/scratch_devs of=/dev/stdout 2023-12-17T12:03:27.486 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-17T12:03:27.487 DEBUG:teuthology.orchestra.run.smithi141:> stat /dev/vg_nvme/lv_1 2023-12-17T12:03:27.573 INFO:teuthology.orchestra.run.smithi141.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-17T12:03:27.573 INFO:teuthology.orchestra.run.smithi141.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:27.573 INFO:teuthology.orchestra.run.smithi141.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-12-17T12:03:27.573 INFO:teuthology.orchestra.run.smithi141.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:27.574 INFO:teuthology.orchestra.run.smithi141.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:27.574 INFO:teuthology.orchestra.run.smithi141.stdout:Access: 2023-12-17 12:03:14.346824081 +0000 2023-12-17T12:03:27.574 INFO:teuthology.orchestra.run.smithi141.stdout:Modify: 2023-12-17 11:51:00.543859408 +0000 2023-12-17T12:03:27.574 INFO:teuthology.orchestra.run.smithi141.stdout:Change: 2023-12-17 11:51:00.543859408 +0000 2023-12-17T12:03:27.574 INFO:teuthology.orchestra.run.smithi141.stdout: Birth: 2023-12-17 11:51:00.514860041 +0000 2023-12-17T12:03:27.574 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-17T12:03:27.640 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records in 2023-12-17T12:03:27.640 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records out 2023-12-17T12:03:27.640 INFO:teuthology.orchestra.run.smithi141.stderr:512 bytes copied, 0.00029893 s, 1.7 MB/s 2023-12-17T12:03:27.643 DEBUG:teuthology.orchestra.run.smithi141:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-17T12:03:27.688 DEBUG:teuthology.orchestra.run.smithi141:> stat /dev/vg_nvme/lv_2 2023-12-17T12:03:27.771 INFO:teuthology.orchestra.run.smithi141.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-17T12:03:27.771 INFO:teuthology.orchestra.run.smithi141.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:27.772 INFO:teuthology.orchestra.run.smithi141.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-12-17T12:03:27.772 INFO:teuthology.orchestra.run.smithi141.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:27.772 INFO:teuthology.orchestra.run.smithi141.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:27.772 INFO:teuthology.orchestra.run.smithi141.stdout:Access: 2023-12-17 12:03:14.346824081 +0000 2023-12-17T12:03:27.772 INFO:teuthology.orchestra.run.smithi141.stdout:Modify: 2023-12-17 11:51:01.384841049 +0000 2023-12-17T12:03:27.772 INFO:teuthology.orchestra.run.smithi141.stdout:Change: 2023-12-17 11:51:01.384841049 +0000 2023-12-17T12:03:27.772 INFO:teuthology.orchestra.run.smithi141.stdout: Birth: 2023-12-17 11:51:01.357841639 +0000 2023-12-17T12:03:27.772 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-17T12:03:27.876 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records in 2023-12-17T12:03:27.876 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records out 2023-12-17T12:03:27.877 INFO:teuthology.orchestra.run.smithi141.stderr:512 bytes copied, 0.000508224 s, 1.0 MB/s 2023-12-17T12:03:27.879 DEBUG:teuthology.orchestra.run.smithi141:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-17T12:03:27.924 DEBUG:teuthology.orchestra.run.smithi141:> stat /dev/vg_nvme/lv_3 2023-12-17T12:03:28.008 INFO:teuthology.orchestra.run.smithi141.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-17T12:03:28.008 INFO:teuthology.orchestra.run.smithi141.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:28.008 INFO:teuthology.orchestra.run.smithi141.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-12-17T12:03:28.009 INFO:teuthology.orchestra.run.smithi141.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:28.009 INFO:teuthology.orchestra.run.smithi141.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:28.009 INFO:teuthology.orchestra.run.smithi141.stdout:Access: 2023-12-17 12:03:14.346824081 +0000 2023-12-17T12:03:28.009 INFO:teuthology.orchestra.run.smithi141.stdout:Modify: 2023-12-17 11:51:02.201823214 +0000 2023-12-17T12:03:28.009 INFO:teuthology.orchestra.run.smithi141.stdout:Change: 2023-12-17 11:51:02.201823214 +0000 2023-12-17T12:03:28.009 INFO:teuthology.orchestra.run.smithi141.stdout: Birth: 2023-12-17 11:51:02.177823738 +0000 2023-12-17T12:03:28.009 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-17T12:03:28.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:27 smithi141 ceph-mon[41225]: Reconfiguring mon.c (monmap changed)... 2023-12-17T12:03:28.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:27 smithi141 ceph-mon[41225]: Reconfiguring daemon mon.c on smithi192 2023-12-17T12:03:28.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:27 smithi141 ceph-mon[41225]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi141=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:28.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:27 smithi141 ceph-mon[41225]: Saving service mgr spec with placement smithi027=x;smithi141=y;smithi192=z;count:3 2023-12-17T12:03:28.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:27 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:28.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:27 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:28.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:27 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:28.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:27 smithi192 ceph-mon[41899]: Reconfiguring mon.c (monmap changed)... 2023-12-17T12:03:28.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:27 smithi192 ceph-mon[41899]: Reconfiguring daemon mon.c on smithi192 2023-12-17T12:03:28.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:27 smithi192 ceph-mon[41899]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi141=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:28.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:27 smithi192 ceph-mon[41899]: Saving service mgr spec with placement smithi027=x;smithi141=y;smithi192=z;count:3 2023-12-17T12:03:28.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:27 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:28.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:27 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:28.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:27 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:28.080 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records in 2023-12-17T12:03:28.080 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records out 2023-12-17T12:03:28.080 INFO:teuthology.orchestra.run.smithi141.stderr:512 bytes copied, 0.000293652 s, 1.7 MB/s 2023-12-17T12:03:28.082 DEBUG:teuthology.orchestra.run.smithi141:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-17T12:03:28.128 DEBUG:teuthology.orchestra.run.smithi141:> stat /dev/vg_nvme/lv_4 2023-12-17T12:03:28.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:27 smithi027 ceph-mon[36871]: Reconfiguring mon.c (monmap changed)... 2023-12-17T12:03:28.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:27 smithi027 ceph-mon[36871]: Reconfiguring daemon mon.c on smithi192 2023-12-17T12:03:28.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:27 smithi027 ceph-mon[36871]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi141=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:28.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:27 smithi027 ceph-mon[36871]: Saving service mgr spec with placement smithi027=x;smithi141=y;smithi192=z;count:3 2023-12-17T12:03:28.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:27 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:28.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:27 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:28.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:27 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout:Access: 2023-12-17 12:03:14.346824081 +0000 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout:Modify: 2023-12-17 11:51:03.008805597 +0000 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout:Change: 2023-12-17 11:51:03.008805597 +0000 2023-12-17T12:03:28.212 INFO:teuthology.orchestra.run.smithi141.stdout: Birth: 2023-12-17 11:51:02.984806121 +0000 2023-12-17T12:03:28.212 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-17T12:03:28.318 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records in 2023-12-17T12:03:28.318 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records out 2023-12-17T12:03:28.318 INFO:teuthology.orchestra.run.smithi141.stderr:512 bytes copied, 0.000333343 s, 1.5 MB/s 2023-12-17T12:03:28.320 DEBUG:teuthology.orchestra.run.smithi141:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-17T12:03:28.365 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-17T12:03:28.366 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2023-12-17T12:03:28.408 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-17T12:03:28.408 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-17 12:03:05.387218079 +0000 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-17 11:50:53.863600155 +0000 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-17 11:50:53.863600155 +0000 2023-12-17T12:03:28.495 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-17 11:50:53.838600640 +0000 2023-12-17T12:03:28.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-17T12:03:28.604 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-17T12:03:28.605 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-17T12:03:28.605 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000315772 s, 1.6 MB/s 2023-12-17T12:03:28.608 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-17T12:03:28.653 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-17 12:03:05.388218059 +0000 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-17 11:50:54.679584334 +0000 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-17 11:50:54.679584334 +0000 2023-12-17T12:03:28.738 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-17 11:50:54.655584799 +0000 2023-12-17T12:03:28.739 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-17T12:03:28.847 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-17T12:03:28.847 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-17T12:03:28.848 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000284417 s, 1.8 MB/s 2023-12-17T12:03:28.848 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:28 smithi192 ceph-mon[41899]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:28.848 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:28.848 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:28.848 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:28.848 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:28.848 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-17T12:03:28.848 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:28.848 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:28.850 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-17T12:03:28.894 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2023-12-17T12:03:28.977 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-17T12:03:28.978 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:28.978 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-12-17T12:03:28.978 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:28.978 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:28.978 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-17 12:03:05.388218059 +0000 2023-12-17T12:03:28.978 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-17 11:50:55.504568339 +0000 2023-12-17T12:03:28.978 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-17 11:50:55.504568339 +0000 2023-12-17T12:03:28.978 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-17 11:50:55.480568804 +0000 2023-12-17T12:03:28.978 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-17T12:03:29.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:28 smithi141 ceph-mon[41225]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:29.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:29.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:29.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:29.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:29.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-17T12:03:29.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:29.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:29.084 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-17T12:03:29.084 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-17T12:03:29.085 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000331518 s, 1.5 MB/s 2023-12-17T12:03:29.087 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-17T12:03:29.133 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2023-12-17T12:03:29.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:28 smithi027 ceph-mon[36871]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:29.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:29.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:29.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:29.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:29.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-17T12:03:29.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:29.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-17 12:03:05.388218059 +0000 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-17 11:50:56.333552265 +0000 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-17 11:50:56.333552265 +0000 2023-12-17T12:03:29.218 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-17 11:50:56.306552789 +0000 2023-12-17T12:03:29.218 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-17T12:03:29.324 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-17T12:03:29.324 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-17T12:03:29.324 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000306883 s, 1.7 MB/s 2023-12-17T12:03:29.327 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-17T12:03:29.372 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2023-12-17T12:03:29.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-17T12:03:29.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:29 smithi192 ceph-mon[41899]: Deploying daemon mgr.z on smithi192 2023-12-17T12:03:29.794 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:03:29.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:29 smithi027 ceph-mon[36871]: Deploying daemon mgr.z on smithi192 2023-12-17T12:03:30.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:29 smithi141 ceph-mon[41225]: Deploying daemon mgr.z on smithi192 2023-12-17T12:03:30.062 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:29 smithi192 systemd[1]: Starting Ceph mgr.z for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:03:30.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mon[41899]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:30 smithi192 bash[44017]: b77d5f30b108586e632dd414a173997b9fd470ec723e6eb5cdcc8b904f3ea4ca 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:30 smithi192 systemd[1]: Started Ceph mgr.z for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:03:30.812 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-mgr[44055]: -- 172.21.15.192:0/2497852333 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c2db03520 con 0x555c2dada800 2023-12-17T12:03:31.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:30 smithi141 ceph-mon[41225]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:31.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:30 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:31.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:30 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:31.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:30 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:31.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:30 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:31.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:30 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-17T12:03:31.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:30 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:31.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:30 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:31.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:30 smithi027 ceph-mon[36871]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:31.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:30 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:31.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:30 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:31.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:30 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:31.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:30 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:31.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:30 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-17T12:03:31.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:30 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:31.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:30 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:31.306 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:30.965+0000 7f8da42b5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-17T12:03:31.306 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:31 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:31.083+0000 7f8da42b5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-17T12:03:31.306 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:31 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:31.194+0000 7f8da42b5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-17T12:03:31.561 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:31 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:31.303+0000 7f8da42b5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-17T12:03:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-17T12:03:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-17T12:03:31.582 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2023-12-17T12:03:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2023-12-17T12:03:31.582 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0308727 s, 340 MB/s 2023-12-17T12:03:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2023-12-17T12:03:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:31 smithi027 ceph-mon[36871]: Deploying daemon mgr.y on smithi141 2023-12-17T12:03:32.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2023-12-17T12:03:32.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:31 smithi141 ceph-mon[41225]: Deploying daemon mgr.y on smithi141 2023-12-17T12:03:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:31 smithi192 ceph-mon[41899]: Deploying daemon mgr.y on smithi141 2023-12-17T12:03:32.421 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:03:32.458 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:32.150+0000 7f8da42b5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:03:32.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:32 smithi141 systemd[1]: Starting Ceph mgr.y for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:03:32.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:32 smithi192 ceph-mon[41899]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:32.709 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:32.456+0000 7f8da42b5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-17T12:03:32.709 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:32.567+0000 7f8da42b5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-17T12:03:32.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:32 smithi027 ceph-mon[36871]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:33.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:32 smithi141 ceph-mon[41225]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:33.061 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:32.707+0000 7f8da42b5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-17T12:03:33.061 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:32.900+0000 7f8da42b5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-17T12:03:33.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:33 smithi141 bash[42652]: 80bf96e71c55322d51c0a6d78955a626e7a81e6a8bdeb622096c271fe0095d4c 2023-12-17T12:03:33.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:33 smithi141 ceph-mgr[42690]: -- 172.21.15.141:0/423765672 <== mon.2 v2:172.21.15.141:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56341b93f520 con 0x56341b916800 2023-12-17T12:03:33.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:33 smithi141 systemd[1]: Started Ceph mgr.y for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:03:33.561 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:33.071+0000 7f8da42b5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-17T12:03:33.561 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:33.183+0000 7f8da42b5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-17T12:03:33.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 -- 172.21.15.27:0/3333669365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102fe0 msgr2=0x7f6864109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:33.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 --2- 172.21.15.27:0/3333669365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102fe0 0x7f6864109890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f685c00b0d0 tx=0x7f685c00b3e0 comp rx=0 tx=0).stop 2023-12-17T12:03:33.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 -- 172.21.15.27:0/3333669365 shutdown_connections 2023-12-17T12:03:33.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 --2- 172.21.15.27:0/3333669365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102fe0 0x7f6864109890 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:33.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 --2- 172.21.15.27:0/3333669365 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6864101440 0x7f6864101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:33.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 --2- 172.21.15.27:0/3333669365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6864102640 0x7f6864102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:33.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 -- 172.21.15.27:0/3333669365 >> 172.21.15.27:0/3333669365 conn(0x7f68640fc9e0 msgr2=0x7f68640fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:33.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 -- 172.21.15.27:0/3333669365 shutdown_connections 2023-12-17T12:03:33.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.566+0000 7f686a557700 1 -- 172.21.15.27:0/3333669365 wait complete. 2023-12-17T12:03:33.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.567+0000 7f686a557700 1 Processor -- start 2023-12-17T12:03:33.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.567+0000 7f686a557700 1 -- start start 2023-12-17T12:03:33.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.567+0000 7f686a557700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6864101440 0x7f6864197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.567+0000 7f686a557700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102640 0x7f6864197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f686a557700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6864102fe0 0x7f686419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f686a557700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686419ee70 con 0x7f6864102640 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f686a557700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686419efb0 con 0x7f6864102fe0 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f686a557700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686419f140 con 0x7f6864101440 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6868d54700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102640 0x7f6864197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6869555700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6864101440 0x7f6864197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6869555700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6864101440 0x7f6864197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:59346/0 (socket says 172.21.15.27:59346) 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6869555700 1 -- 172.21.15.27:0/2788415170 learned_addr learned my addr 172.21.15.27:0/2788415170 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6869d56700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6864102fe0 0x7f686419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6868d54700 1 -- 172.21.15.27:0/2788415170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6864101440 msgr2=0x7f6864197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6868d54700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6864101440 0x7f6864197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:33.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6868d54700 1 -- 172.21.15.27:0/2788415170 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6864102fe0 msgr2=0x7f686419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:33.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6868d54700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6864102fe0 0x7f686419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:33.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6868d54700 1 -- 172.21.15.27:0/2788415170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c009d00 con 0x7f6864102640 2023-12-17T12:03:33.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.568+0000 7f6869555700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6864101440 0x7f6864197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:03:33.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.569+0000 7f6868d54700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102640 0x7f6864197870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f685400ea20 tx=0x7f685400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:33.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.569+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400cbd0 con 0x7f6864102640 2023-12-17T12:03:33.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.569+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f685400ee60 con 0x7f6864102640 2023-12-17T12:03:33.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.569+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686419f420 con 0x7f6864102640 2023-12-17T12:03:33.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.569+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854018420 con 0x7f6864102640 2023-12-17T12:03:33.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.569+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686419f970 con 0x7f6864102640 2023-12-17T12:03:33.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.570+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6864108940 con 0x7f6864102640 2023-12-17T12:03:33.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.570+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f6854010430 con 0x7f6864102640 2023-12-17T12:03:33.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.571+0000 7f685a7fc700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6850038af0 0x7f685003afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:33.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.571+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6854014070 con 0x7f6864102640 2023-12-17T12:03:33.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.571+0000 7f6869555700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6850038af0 0x7f685003afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:33.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.572+0000 7f6869555700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6850038af0 0x7f685003afb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6860005950 tx=0x7f6860009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:33.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.576+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6854016330 con 0x7f6864102640 2023-12-17T12:03:33.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:33.800+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6864061a10 con 0x7f6850038af0 2023-12-17T12:03:33.881 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:33.518+0000 7fe4d6e26200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-17T12:03:33.881 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:33.645+0000 7fe4d6e26200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-17T12:03:33.881 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:33.764+0000 7fe4d6e26200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-17T12:03:34.061 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:33.684+0000 7f8da42b5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-17T12:03:34.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:34.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:03:34.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:03:34.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:34.445 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:34.063+0000 7f8da42b5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-17T12:03:34.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:34.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:03:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:03:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:34.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:34.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:34.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:34.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:03:34.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:03:34.517 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:34.517 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:33.879+0000 7fe4d6e26200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-17T12:03:34.811 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:34.443+0000 7f8da42b5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-17T12:03:34.811 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:34.572+0000 7f8da42b5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-17T12:03:35.111 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:34 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:34.803+0000 7fe4d6e26200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:03:35.363 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:35.109+0000 7fe4d6e26200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-17T12:03:35.363 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:35.221+0000 7fe4d6e26200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-17T12:03:35.363 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:35.363 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.363 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.363 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.364 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.364 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:35.364 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:35.364 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.364 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:35.364 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:35.364 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:35.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:35.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:35.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:35 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:35.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:35.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:35.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:35.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:35.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-17T12:03:35.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-17T12:03:35.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:35.726 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:35.361+0000 7fe4d6e26200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-17T12:03:35.726 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:35.555+0000 7fe4d6e26200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-17T12:03:35.811 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:35.559+0000 7f8da42b5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-17T12:03:35.811 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:35.681+0000 7f8da42b5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-17T12:03:36.015 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:35.724+0000 7fe4d6e26200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-17T12:03:36.015 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:35.836+0000 7fe4d6e26200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-17T12:03:36.144 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:35.874+0000 7f8da42b5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-17T12:03:36.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:36 smithi027 ceph-mon[36871]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:36 smithi027 ceph-mon[36871]: Reconfiguring mgr.x (unknown last config time)... 2023-12-17T12:03:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:36 smithi027 ceph-mon[36871]: Reconfiguring daemon mgr.x on smithi027 2023-12-17T12:03:36.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:36.315+0000 7fe4d6e26200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-17T12:03:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:36 smithi141 ceph-mon[41225]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:36 smithi141 ceph-mon[41225]: Reconfiguring mgr.x (unknown last config time)... 2023-12-17T12:03:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:36 smithi141 ceph-mon[41225]: Reconfiguring daemon mgr.x on smithi027 2023-12-17T12:03:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:36 smithi192 ceph-mon[41899]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:36 smithi192 ceph-mon[41899]: Reconfiguring mgr.x (unknown last config time)... 2023-12-17T12:03:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:36 smithi192 ceph-mon[41899]: Reconfiguring daemon mgr.x on smithi027 2023-12-17T12:03:37.015 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:36.664+0000 7fe4d6e26200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-17T12:03:37.061 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:36.686+0000 7f8da42b5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-17T12:03:37.061 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:36.918+0000 7f8da42b5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-17T12:03:37.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:37 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2598416931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]: dispatch 2023-12-17T12:03:37.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:37 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]: dispatch 2023-12-17T12:03:37.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:37 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]': finished 2023-12-17T12:03:37.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:37 smithi027 ceph-mon[36871]: osdmap e5: 1 total, 0 up, 1 in 2023-12-17T12:03:37.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:37 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:37.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:37.017+0000 7fe4d6e26200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-17T12:03:37.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:37.138+0000 7fe4d6e26200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-17T12:03:37.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:37 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2598416931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]: dispatch 2023-12-17T12:03:37.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:37 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]: dispatch 2023-12-17T12:03:37.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:37 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]': finished 2023-12-17T12:03:37.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:37 smithi141 ceph-mon[41225]: osdmap e5: 1 total, 0 up, 1 in 2023-12-17T12:03:37.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:37 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:37.561 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:37 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:37.169+0000 7f8da42b5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-17T12:03:37.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:37 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2598416931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]: dispatch 2023-12-17T12:03:37.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:37 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]: dispatch 2023-12-17T12:03:37.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:37 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fb8ca94-6c9e-452e-a99f-3b84eb305dc1"}]': finished 2023-12-17T12:03:37.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:37 smithi192 ceph-mon[41899]: osdmap e5: 1 total, 0 up, 1 in 2023-12-17T12:03:37.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:37 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:38.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:38 smithi027 ceph-mon[36871]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:38.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:38 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2043869989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:03:38.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:38 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:38.416 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:38.091+0000 7fe4d6e26200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-17T12:03:38.416 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:38.216+0000 7fe4d6e26200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-17T12:03:38.416 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:38 smithi141 ceph-mon[41225]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:38.416 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:38 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2043869989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:03:38.416 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:38 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:38.561 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:38.298+0000 7f8da42b5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-17T12:03:38.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:38 smithi192 ceph-mon[41899]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:38.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:38 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2043869989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:03:38.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:38 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:38.765 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:38.413+0000 7fe4d6e26200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-17T12:03:39.018 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:38.648+0000 7f8da42b5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:03:39.018 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:38.765+0000 7f8da42b5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-17T12:03:39.311 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:03:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z[44032]: 2023-12-17T12:03:39.016+0000 7f8da42b5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-17T12:03:39.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:39.246+0000 7fe4d6e26200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-17T12:03:39.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:39.480+0000 7fe4d6e26200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-17T12:03:39.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:39.794+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 14) v1 ==== 90793+0+0 (secure 0 0 0) 0x7f68540107e0 con 0x7f6864102640 2023-12-17T12:03:40.015 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:39.731+0000 7fe4d6e26200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-17T12:03:40.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-mon[41225]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:40.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-mon[41225]: Standby manager daemon z started 2023-12-17T12:03:40.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-mon[41225]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-17T12:03:40.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-mon[41225]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-17T12:03:40.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-mon[41225]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-17T12:03:40.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:39 smithi141 ceph-mon[41225]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-17T12:03:40.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:39 smithi192 ceph-mon[41899]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:40.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:39 smithi192 ceph-mon[41899]: Standby manager daemon z started 2023-12-17T12:03:40.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:39 smithi192 ceph-mon[41899]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-17T12:03:40.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:39 smithi192 ceph-mon[41899]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-17T12:03:40.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:39 smithi192 ceph-mon[41899]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-17T12:03:40.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:39 smithi192 ceph-mon[41899]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-17T12:03:40.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:39 smithi027 ceph-mon[36871]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:40.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:39 smithi027 ceph-mon[36871]: Standby manager daemon z started 2023-12-17T12:03:40.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:39 smithi027 ceph-mon[36871]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-17T12:03:40.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:39 smithi027 ceph-mon[36871]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-17T12:03:40.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:39 smithi027 ceph-mon[36871]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-17T12:03:40.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:39 smithi027 ceph-mon[36871]: from='mgr.? 172.21.15.192:0/3447279186' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-17T12:03:41.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:40 smithi027 ceph-mon[36871]: mgrmap e14: x(active, since 77s), standbys: z 2023-12-17T12:03:41.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-17T12:03:41.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:40 smithi192 ceph-mon[41899]: mgrmap e14: x(active, since 77s), standbys: z 2023-12-17T12:03:41.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-17T12:03:41.214 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:40 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:40.865+0000 7fe4d6e26200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-17T12:03:41.214 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:40 smithi141 ceph-mon[41225]: mgrmap e14: x(active, since 77s), standbys: z 2023-12-17T12:03:41.215 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-17T12:03:41.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:41.212+0000 7fe4d6e26200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-17T12:03:41.515 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:41.327+0000 7fe4d6e26200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-17T12:03:42.016 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:03:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y[42667]: 2023-12-17T12:03:41.576+0000 7fe4d6e26200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-17T12:03:42.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:42.582+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f685401f310 con 0x7f6864102640 2023-12-17T12:03:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: Standby manager daemon y started 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:42.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: Standby manager daemon y started 2023-12-17T12:03:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-17T12:03:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-17T12:03:43.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-17T12:03:43.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-17T12:03:43.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:43.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:43.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:43.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: Standby manager daemon y started 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.? 172.21.15.141:0/3421205245' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:43.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:43.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:43 smithi027 ceph-mon[36871]: mgrmap e15: x(active, since 79s), standbys: y, z 2023-12-17T12:03:43.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-17T12:03:43.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-17T12:03:43.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:44.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:43 smithi141 ceph-mon[41225]: mgrmap e15: x(active, since 79s), standbys: y, z 2023-12-17T12:03:44.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:43 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-17T12:03:44.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:43 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-17T12:03:44.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:43 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:44.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:43 smithi192 ceph-mon[41899]: mgrmap e15: x(active, since 79s), standbys: y, z 2023-12-17T12:03:44.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:43 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-17T12:03:44.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:43 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-17T12:03:44.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:43 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:44.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:44 smithi027 ceph-mon[36871]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:44.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:44 smithi027 ceph-mon[36871]: Deploying daemon osd.0 on smithi027 2023-12-17T12:03:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:44 smithi141 ceph-mon[41225]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:44 smithi141 ceph-mon[41225]: Deploying daemon osd.0 on smithi027 2023-12-17T12:03:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:44 smithi192 ceph-mon[41899]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:44 smithi192 ceph-mon[41899]: Deploying daemon osd.0 on smithi027 2023-12-17T12:03:46.854 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:46 smithi027 ceph-mon[36871]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:47.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:46 smithi141 ceph-mon[41225]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:47.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:46 smithi192 ceph-mon[41899]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:48.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:48 smithi027 ceph-mon[36871]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:49.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:48 smithi141 ceph-mon[41225]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:48 smithi192 ceph-mon[41899]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:50.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:50 smithi027 ceph-mon[36871]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:50.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:50.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:50.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:50.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:50 smithi141 ceph-mon[41225]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:50.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:50.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:50.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:50.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:50 smithi192 ceph-mon[41899]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:50.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:50.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:50.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:52.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:51 smithi027 ceph-mon[36871]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:52.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:52.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:52.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:51 smithi141 ceph-mon[41225]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:52.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:52.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:52.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:51 smithi192 ceph-mon[41899]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:52.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:52.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:52.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.355+0000 7f685a7fc700 1 -- 172.21.15.27:0/2788415170 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6864061a10 con 0x7f6850038af0 2023-12-17T12:03:52.358 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2023-12-17T12:03:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.357+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6850038af0 msgr2=0x7f685003afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.357+0000 7f686a557700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6850038af0 0x7f685003afb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6860005950 tx=0x7f6860009260 comp rx=0 tx=0).stop 2023-12-17T12:03:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.357+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102640 msgr2=0x7f6864197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.357+0000 7f686a557700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102640 0x7f6864197870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f685400ea20 tx=0x7f685400ed30 comp rx=0 tx=0).stop 2023-12-17T12:03:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.357+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 shutdown_connections 2023-12-17T12:03:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.358+0000 7f686a557700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6850038af0 0x7f685003afb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.358+0000 7f686a557700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6864101440 0x7f6864197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.358+0000 7f686a557700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6864102fe0 0x7f686419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.358+0000 7f686a557700 1 --2- 172.21.15.27:0/2788415170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6864102640 0x7f6864197870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.358+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 >> 172.21.15.27:0/2788415170 conn(0x7f68640fc9e0 msgr2=0x7f6864105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.358+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 shutdown_connections 2023-12-17T12:03:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:03:52.358+0000 7f686a557700 1 -- 172.21.15.27:0/2788415170 wait complete. 2023-12-17T12:03:52.745 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.0.service 2023-12-17T12:03:52.749 INFO:tasks.cephadm:Deploying osd.1 on smithi141 with /dev/vg_nvme/lv_4... 2023-12-17T12:03:52.749 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-17T12:03:52.829 INFO:journalctl@ceph.osd.0.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:03:53.153 INFO:teuthology.orchestra.run.smithi141.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.b/config 2023-12-17T12:03:53.536 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:03:53 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-0[55535]: 2023-12-17T12:03:53.250+0000 7f6cd09f3640 -1 osd.0 0 log_to_monitors true 2023-12-17T12:03:53.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:53 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:53.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:53 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:53.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:53 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:53.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:53 smithi027 ceph-mon[36871]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-17T12:03:53.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:53 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:53.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:53 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:53.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:53 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:53.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:53 smithi192 ceph-mon[41899]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-17T12:03:53.664 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:53 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:53.664 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:53 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:53.664 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:53 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:03:53.664 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:53 smithi141 ceph-mon[41225]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-17T12:03:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:54 smithi141 ceph-mon[41225]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:54 smithi141 ceph-mon[41225]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-17T12:03:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:54 smithi141 ceph-mon[41225]: osdmap e6: 1 total, 0 up, 1 in 2023-12-17T12:03:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:54 smithi141 ceph-mon[41225]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:03:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:54.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:54 smithi027 ceph-mon[36871]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:54.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:54 smithi027 ceph-mon[36871]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-17T12:03:54.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:54 smithi027 ceph-mon[36871]: osdmap e6: 1 total, 0 up, 1 in 2023-12-17T12:03:54.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:54 smithi027 ceph-mon[36871]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:03:54.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:54.664 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:03:54 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-0[55535]: 2023-12-17T12:03:54.337+0000 7f6cc606a700 -1 osd.0 0 waiting for initial osdmap 2023-12-17T12:03:54.664 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:03:54 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-0[55535]: 2023-12-17T12:03:54.341+0000 7f6cbfe60700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:03:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:54 smithi192 ceph-mon[41899]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:54 smithi192 ceph-mon[41899]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-17T12:03:54.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:54 smithi192 ceph-mon[41899]: osdmap e6: 1 total, 0 up, 1 in 2023-12-17T12:03:54.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:54 smithi192 ceph-mon[41899]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:03:54.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:54.888 INFO:teuthology.orchestra.run.smithi141.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-17T12:03:54.888 INFO:teuthology.orchestra.run.smithi141.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-17T12:03:54.888 INFO:teuthology.orchestra.run.smithi141.stderr: stderr: 10+0 records in 2023-12-17T12:03:54.889 INFO:teuthology.orchestra.run.smithi141.stderr:10+0 records out 2023-12-17T12:03:54.889 INFO:teuthology.orchestra.run.smithi141.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0302724 s, 346 MB/s 2023-12-17T12:03:54.889 INFO:teuthology.orchestra.run.smithi141.stderr:--> Zapping successful for: 2023-12-17T12:03:55.283 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi141:vg_nvme/lv_4 2023-12-17T12:03:55.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:55 smithi141 ceph-mon[41225]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:03:55.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:55 smithi141 ceph-mon[41225]: osdmap e7: 1 total, 0 up, 1 in 2023-12-17T12:03:55.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:55.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:55.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:55 smithi027 ceph-mon[36871]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:03:55.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:55 smithi027 ceph-mon[36871]: osdmap e7: 1 total, 0 up, 1 in 2023-12-17T12:03:55.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:55.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:55.683 INFO:teuthology.orchestra.run.smithi141.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.b/config 2023-12-17T12:03:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:55 smithi192 ceph-mon[41899]: from='osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:03:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:55 smithi192 ceph-mon[41899]: osdmap e7: 1 total, 0 up, 1 in 2023-12-17T12:03:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] boot 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: osdmap e8: 1 total, 1 up, 1 in 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:56.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:56 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] boot 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: osdmap e8: 1 total, 1 up, 1 in 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:56.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:56 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:03:56.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:03:56.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-17T12:03:56.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: osd.0 [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] boot 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: osdmap e8: 1 total, 1 up, 1 in 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:03:56.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:56 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 -- 172.21.15.141:0/298152064 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 msgr2=0x7ff014109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 --2- 172.21.15.141:0/298152064 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 0x7ff014109a70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00b0d0 tx=0x7ff00c00b3e0 comp rx=0 tx=0).stop 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 -- 172.21.15.141:0/298152064 shutdown_connections 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 --2- 172.21.15.141:0/298152064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff014101620 0x7ff014101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 --2- 172.21.15.141:0/298152064 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 0x7ff014109a70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 --2- 172.21.15.141:0/298152064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff014102820 0x7ff014102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 -- 172.21.15.141:0/298152064 >> 172.21.15.141:0/298152064 conn(0x7ff0140fcbe0 msgr2=0x7ff0140ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 -- 172.21.15.141:0/298152064 shutdown_connections 2023-12-17T12:03:56.868 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.865+0000 7ff019a9f700 1 -- 172.21.15.141:0/298152064 wait complete. 2023-12-17T12:03:56.869 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.866+0000 7ff019a9f700 1 Processor -- start 2023-12-17T12:03:56.869 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.866+0000 7ff019a9f700 1 -- start start 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff019a9f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff014101620 0x7ff0141975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff019a9f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff014102820 0x7ff014197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff019a9f700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 0x7ff01419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff019a9f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419f0e0 con 0x7ff014102820 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff019a9f700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419f220 con 0x7ff0141031c0 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff019a9f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419f3b0 con 0x7ff014101620 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff013fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 0x7ff01419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff013fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 0x7ff01419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:33258/0 (socket says 172.21.15.141:33258) 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff013fff700 1 -- 172.21.15.141:0/1722668624 learned_addr learned my addr 172.21.15.141:0/1722668624 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff0137fe700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff014101620 0x7ff0141975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff012ffd700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff014102820 0x7ff014197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff013fff700 1 -- 172.21.15.141:0/1722668624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff014101620 msgr2=0x7ff0141975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:56.871 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff013fff700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff014101620 0x7ff0141975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff013fff700 1 -- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff014102820 msgr2=0x7ff014197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff013fff700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff014102820 0x7ff014197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff013fff700 1 -- 172.21.15.141:0/1722668624 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c009d00 con 0x7ff0141031c0 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff012ffd700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff014102820 0x7ff014197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.867+0000 7ff0137fe700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff014101620 0x7ff0141975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.868+0000 7ff013fff700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 0x7ff01419eb10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff00c007b60 tx=0x7ff00c0094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.868+0000 7ff010ff9700 1 -- 172.21.15.141:0/1722668624 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c00e040 con 0x7ff0141031c0 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.868+0000 7ff010ff9700 1 -- 172.21.15.141:0/1722668624 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff00c012e10 con 0x7ff0141031c0 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.868+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01419f630 con 0x7ff0141031c0 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.868+0000 7ff010ff9700 1 -- 172.21.15.141:0/1722668624 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c01b450 con 0x7ff0141031c0 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.868+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01419faa0 con 0x7ff0141031c0 2023-12-17T12:03:56.872 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.869+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff014108ad0 con 0x7ff0141031c0 2023-12-17T12:03:56.874 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.871+0000 7ff010ff9700 1 -- 172.21.15.141:0/1722668624 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7ff00c012980 con 0x7ff0141031c0 2023-12-17T12:03:56.875 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.872+0000 7ff010ff9700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7feffc0a13f0 0x7feffc0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:03:56.875 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.873+0000 7ff010ff9700 1 -- 172.21.15.141:0/1722668624 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7ff00c0cd990 con 0x7ff0141031c0 2023-12-17T12:03:56.876 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.873+0000 7ff0137fe700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7feffc0a13f0 0x7feffc0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:03:56.876 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.874+0000 7ff0137fe700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7feffc0a13f0 0x7feffc0a38b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff014102680 tx=0x7ff00400b670 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:03:56.878 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:56.876+0000 7ff010ff9700 1 -- 172.21.15.141:0/1722668624 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff00c098a50 con 0x7ff0141031c0 2023-12-17T12:03:57.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:03:57.084+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff01402d3b0 con 0x7feffc0a13f0 2023-12-17T12:03:57.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:57 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi027 2023-12-17T12:03:57.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:57 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi027 to 17271M 2023-12-17T12:03:57.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:57 smithi027 ceph-mon[36871]: osdmap e9: 1 total, 1 up, 1 in 2023-12-17T12:03:57.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:57 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:03:57.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:57 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:03:57.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:57 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:57.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:57 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi027 2023-12-17T12:03:57.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:57 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi027 to 17271M 2023-12-17T12:03:57.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:57 smithi141 ceph-mon[41225]: osdmap e9: 1 total, 1 up, 1 in 2023-12-17T12:03:57.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:57 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:03:57.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:57 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:03:57.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:57 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:57.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:57 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi027 2023-12-17T12:03:57.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:57 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi027 to 17271M 2023-12-17T12:03:57.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:57 smithi192 ceph-mon[41899]: osdmap e9: 1 total, 1 up, 1 in 2023-12-17T12:03:57.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:57 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:03:57.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:57 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:03:57.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:57 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:03:58.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:58 smithi027 ceph-mon[36871]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:03:58.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:03:58 smithi027 ceph-mon[36871]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:58.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:58 smithi141 ceph-mon[41225]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:03:58.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:03:58 smithi141 ceph-mon[41225]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:03:58.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:58 smithi192 ceph-mon[41899]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:03:58.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:03:58 smithi192 ceph-mon[41899]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:04:00.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:00 smithi027 ceph-mon[36871]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:00.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:00 smithi027 ceph-mon[36871]: from='client.? 172.21.15.141:0/47394553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]: dispatch 2023-12-17T12:04:00.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:00 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]: dispatch 2023-12-17T12:04:00.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:00 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]': finished 2023-12-17T12:04:00.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:00 smithi027 ceph-mon[36871]: osdmap e10: 2 total, 1 up, 2 in 2023-12-17T12:04:00.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:00 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:00.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:00 smithi027 ceph-mon[36871]: from='client.? 172.21.15.141:0/2293011174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:00.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:00 smithi141 ceph-mon[41225]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:00.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:00 smithi141 ceph-mon[41225]: from='client.? 172.21.15.141:0/47394553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]: dispatch 2023-12-17T12:04:00.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:00 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]: dispatch 2023-12-17T12:04:00.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:00 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]': finished 2023-12-17T12:04:00.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:00 smithi141 ceph-mon[41225]: osdmap e10: 2 total, 1 up, 2 in 2023-12-17T12:04:00.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:00 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:00.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:00 smithi141 ceph-mon[41225]: from='client.? 172.21.15.141:0/2293011174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:00.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:00 smithi192 ceph-mon[41899]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:00.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:00 smithi192 ceph-mon[41899]: from='client.? 172.21.15.141:0/47394553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]: dispatch 2023-12-17T12:04:00.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:00 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]: dispatch 2023-12-17T12:04:00.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:00 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f6fafe1-a95f-4916-9c2f-545e8be564ae"}]': finished 2023-12-17T12:04:00.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:00 smithi192 ceph-mon[41899]: osdmap e10: 2 total, 1 up, 2 in 2023-12-17T12:04:00.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:00 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:00.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:00 smithi192 ceph-mon[41899]: from='client.? 172.21.15.141:0/2293011174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:02.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:02 smithi027 ceph-mon[36871]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:02.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:02 smithi141 ceph-mon[41225]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:02.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:02 smithi192 ceph-mon[41899]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:04.608 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:04 smithi141 ceph-mon[41225]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:04.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:04 smithi027 ceph-mon[36871]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:04.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:04 smithi192 ceph-mon[41899]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:06.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:06 smithi027 ceph-mon[36871]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:06.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-17T12:04:06.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:06.758 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:06 smithi141 ceph-mon[41225]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:06.759 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-17T12:04:06.759 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:06 smithi192 ceph-mon[41899]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:06.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-17T12:04:06.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:07.623 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:07 smithi141 ceph-mon[41225]: Deploying daemon osd.1 on smithi141 2023-12-17T12:04:07.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:07 smithi027 ceph-mon[36871]: Deploying daemon osd.1 on smithi141 2023-12-17T12:04:07.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:07 smithi192 ceph-mon[41899]: Deploying daemon osd.1 on smithi141 2023-12-17T12:04:08.597 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:08 smithi141 ceph-mon[41225]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:08.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:08 smithi027 ceph-mon[36871]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:08.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:08 smithi192 ceph-mon[41899]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:10.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:10 smithi027 ceph-mon[36871]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:10.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:10 smithi141 ceph-mon[41225]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:10.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:10 smithi192 ceph-mon[41899]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:12.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:12 smithi027 ceph-mon[36871]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:12.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:12.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:12.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:12.488 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:12 smithi141 ceph-mon[41225]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:12.488 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:12.488 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:12.488 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:12.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:12 smithi192 ceph-mon[41899]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:12.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:12.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:12.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:14.008 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.006+0000 7ff010ff9700 1 -- 172.21.15.141:0/1722668624 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff01402d3b0 con 0x7feffc0a13f0 2023-12-17T12:04:14.008 INFO:teuthology.orchestra.run.smithi141.stdout:Created osd(s) 1 on host 'smithi141' 2023-12-17T12:04:14.010 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7feffc0a13f0 msgr2=0x7feffc0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7feffc0a13f0 0x7feffc0a38b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff014102680 tx=0x7ff00400b670 comp rx=0 tx=0).stop 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 msgr2=0x7ff01419eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 0x7ff01419eb10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff00c007b60 tx=0x7ff00c0094f0 comp rx=0 tx=0).stop 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 shutdown_connections 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7feffc0a13f0 0x7feffc0a38b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff014101620 0x7ff0141975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff0141031c0 0x7ff01419eb10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 --2- 172.21.15.141:0/1722668624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff014102820 0x7ff014197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.008+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 >> 172.21.15.141:0/1722668624 conn(0x7ff0140fcbe0 msgr2=0x7ff014105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:04:14.011 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.009+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 shutdown_connections 2023-12-17T12:04:14.012 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:04:14.009+0000 7ff019a9f700 1 -- 172.21.15.141:0/1722668624 wait complete. 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:14.203 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:14.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:14.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:14.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:14.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.414 DEBUG:teuthology.orchestra.run.smithi141:osd.1> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.1.service 2023-12-17T12:04:14.420 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2023-12-17T12:04:14.421 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-17T12:04:14.489 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:14.489 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.489 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.489 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:14.489 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:14.489 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.489 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.490 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:14.490 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:14.503 INFO:journalctl@ceph.osd.1.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:04:14.826 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:04:15.463 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:15 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-1[46239]: 2023-12-17T12:04:15.186+0000 7f7e5f083640 -1 osd.1 0 log_to_monitors true 2023-12-17T12:04:16.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:16 smithi027 ceph-mon[36871]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:16.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:16 smithi027 ceph-mon[36871]: from='osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-17T12:04:16.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:16 smithi027 ceph-mon[36871]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-17T12:04:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:16 smithi141 ceph-mon[41225]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:16 smithi141 ceph-mon[41225]: from='osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-17T12:04:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:16 smithi141 ceph-mon[41225]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-17T12:04:16.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:16 smithi192 ceph-mon[41899]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:16.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:16 smithi192 ceph-mon[41899]: from='osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-17T12:04:16.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:16 smithi192 ceph-mon[41899]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-17T12:04:16.667 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-17T12:04:16.667 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-17T12:04:16.667 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-12-17T12:04:16.667 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-12-17T12:04:16.667 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.030332 s, 346 MB/s 2023-12-17T12:04:16.667 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-12-17T12:04:17.080 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2023-12-17T12:04:17.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:17 smithi192 ceph-mon[41899]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-17T12:04:17.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:17 smithi192 ceph-mon[41899]: osdmap e11: 2 total, 1 up, 2 in 2023-12-17T12:04:17.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:17 smithi192 ceph-mon[41899]: from='osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:04:17.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:17 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:17.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:17 smithi192 ceph-mon[41899]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:04:17.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:17 smithi027 ceph-mon[36871]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-17T12:04:17.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:17 smithi027 ceph-mon[36871]: osdmap e11: 2 total, 1 up, 2 in 2023-12-17T12:04:17.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:17 smithi027 ceph-mon[36871]: from='osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:04:17.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:17 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:17.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:17 smithi027 ceph-mon[36871]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:04:17.483 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:04:17.515 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-1[46239]: 2023-12-17T12:04:17.131+0000 7f7e546fa700 -1 osd.1 0 waiting for initial osdmap 2023-12-17T12:04:17.515 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-1[46239]: 2023-12-17T12:04:17.136+0000 7f7e4e4f0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:04:17.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:17 smithi141 ceph-mon[41225]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-17T12:04:17.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:17 smithi141 ceph-mon[41225]: osdmap e11: 2 total, 1 up, 2 in 2023-12-17T12:04:17.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:17 smithi141 ceph-mon[41225]: from='osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:04:17.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:17 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:17.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:17 smithi141 ceph-mon[41225]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: osdmap e12: 2 total, 1 up, 2 in 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi141 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi141 to 17271M 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:18.313 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:18.313 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:18.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:04:18.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: osdmap e12: 2 total, 1 up, 2 in 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi141 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi141 to 17271M 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:18.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: osdmap e12: 2 total, 1 up, 2 in 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi141 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:04:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi141 to 17271M 2023-12-17T12:04:18.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:18.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:18.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:18.659 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 -- 172.21.15.192:0/3544669443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8101620 msgr2=0x7fbfe8101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:18.659 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/3544669443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8101620 0x7fbfe8101a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009990 tx=0x7fbfe4009ca0 comp rx=0 tx=0).stop 2023-12-17T12:04:18.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 -- 172.21.15.192:0/3544669443 shutdown_connections 2023-12-17T12:04:18.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/3544669443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8101620 0x7fbfe8101a40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:18.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/3544669443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe81031c0 0x7fbfe8109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:18.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/3544669443 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbfe8102820 0x7fbfe8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:18.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 -- 172.21.15.192:0/3544669443 >> 172.21.15.192:0/3544669443 conn(0x7fbfe80fcbe0 msgr2=0x7fbfe80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:04:18.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 -- 172.21.15.192:0/3544669443 shutdown_connections 2023-12-17T12:04:18.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.657+0000 7fbfef5c6700 1 -- 172.21.15.192:0/3544669443 wait complete. 2023-12-17T12:04:18.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.658+0000 7fbfef5c6700 1 Processor -- start 2023-12-17T12:04:18.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfef5c6700 1 -- start start 2023-12-17T12:04:18.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfef5c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe8101620 0x7fbfe8197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:04:18.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfef5c6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102820 0x7fbfe8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:04:18.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfef5c6700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbfe81031c0 0x7fbfe819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:04:18.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfef5c6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe819f020 con 0x7fbfe8101620 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfef5c6700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe819f160 con 0x7fbfe81031c0 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfef5c6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe819f2f0 con 0x7fbfe8102820 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfecb61700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102820 0x7fbfe8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfecb61700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102820 0x7fbfe8197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:55332/0 (socket says 172.21.15.192:55332) 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfecb61700 1 -- 172.21.15.192:0/566219958 learned_addr learned my addr 172.21.15.192:0/566219958 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfed362700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe8101620 0x7fbfe8197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfecb61700 1 -- 172.21.15.192:0/566219958 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbfe81031c0 msgr2=0x7fbfe819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfedb63700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbfe81031c0 0x7fbfe819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.659+0000 7fbfecb61700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbfe81031c0 0x7fbfe819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfecb61700 1 -- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe8101620 msgr2=0x7fbfe8197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfecb61700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe8101620 0x7fbfe8197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfecb61700 1 -- 172.21.15.192:0/566219958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40095f0 con 0x7fbfe8102820 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfedb63700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbfe81031c0 0x7fbfe819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfecb61700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102820 0x7fbfe8197a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800e9d0 tx=0x7fbfd800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfde7fc700 1 -- 172.21.15.192:0/566219958 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8009790 con 0x7fbfe8102820 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe819f5d0 con 0x7fbfe8102820 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfde7fc700 1 -- 172.21.15.192:0/566219958 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfd800cbd0 con 0x7fbfe8102820 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfde7fc700 1 -- 172.21.15.192:0/566219958 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8018680 con 0x7fbfe8102820 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.660+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe819fad0 con 0x7fbfe8102820 2023-12-17T12:04:18.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.661+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe8108ba0 con 0x7fbfe8102820 2023-12-17T12:04:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.665+0000 7fbfde7fc700 1 -- 172.21.15.192:0/566219958 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7fbfd800cd40 con 0x7fbfe8102820 2023-12-17T12:04:18.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.666+0000 7fbfde7fc700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbfd40a13f0 0x7fbfd40a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:04:18.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.666+0000 7fbfde7fc700 1 -- 172.21.15.192:0/566219958 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fbfd8014070 con 0x7fbfe8102820 2023-12-17T12:04:18.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.667+0000 7fbfed362700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbfd40a13f0 0x7fbfd40a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:04:18.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.667+0000 7fbfed362700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbfd40a13f0 0x7fbfd40a38b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b2e0 tx=0x7fbfe400b6b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:04:18.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.669+0000 7fbfde7fc700 1 -- 172.21.15.192:0/566219958 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbfd8096960 con 0x7fbfe8102820 2023-12-17T12:04:18.878 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:18.876+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfe802d380 con 0x7fbfd40a13f0 2023-12-17T12:04:19.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:19 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:04:19.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:19 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:04:19.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:19 smithi027 ceph-mon[36871]: osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] boot 2023-12-17T12:04:19.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:19 smithi027 ceph-mon[36871]: osdmap e13: 2 total, 2 up, 2 in 2023-12-17T12:04:19.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:19.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:04:19.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:04:19.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:19.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:19 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:04:19.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:19 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:04:19.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:19 smithi141 ceph-mon[41225]: osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] boot 2023-12-17T12:04:19.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:19 smithi141 ceph-mon[41225]: osdmap e13: 2 total, 2 up, 2 in 2023-12-17T12:04:19.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:19.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:04:19.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:04:19.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:19.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:19 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:04:19.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:19 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:04:19.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:19 smithi192 ceph-mon[41899]: osd.1 [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] boot 2023-12-17T12:04:19.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:19 smithi192 ceph-mon[41899]: osdmap e13: 2 total, 2 up, 2 in 2023-12-17T12:04:19.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-17T12:04:19.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:04:19.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:04:19.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:20.303 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:20 smithi192 ceph-mon[41899]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:20 smithi192 ceph-mon[41899]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:04:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:20 smithi192 ceph-mon[41899]: osdmap e14: 2 total, 2 up, 2 in 2023-12-17T12:04:20.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:20 smithi027 ceph-mon[36871]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:20.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:20 smithi027 ceph-mon[36871]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:04:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:20 smithi027 ceph-mon[36871]: osdmap e14: 2 total, 2 up, 2 in 2023-12-17T12:04:20.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:20 smithi141 ceph-mon[41225]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-17T12:04:20.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:20 smithi141 ceph-mon[41225]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:04:20.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:20 smithi141 ceph-mon[41225]: osdmap e14: 2 total, 2 up, 2 in 2023-12-17T12:04:22.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:22 smithi027 ceph-mon[36871]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:22.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:22 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/1419005643' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]: dispatch 2023-12-17T12:04:22.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:22 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]: dispatch 2023-12-17T12:04:22.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:22 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]': finished 2023-12-17T12:04:22.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:22 smithi027 ceph-mon[36871]: osdmap e15: 3 total, 2 up, 3 in 2023-12-17T12:04:22.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:22.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:22 smithi141 ceph-mon[41225]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:22 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/1419005643' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]: dispatch 2023-12-17T12:04:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:22 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]: dispatch 2023-12-17T12:04:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:22 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]': finished 2023-12-17T12:04:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:22 smithi141 ceph-mon[41225]: osdmap e15: 3 total, 2 up, 3 in 2023-12-17T12:04:22.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:22.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:22 smithi192 ceph-mon[41899]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:22.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:22 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/1419005643' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]: dispatch 2023-12-17T12:04:22.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:22 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]: dispatch 2023-12-17T12:04:22.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:22 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35099caf-f95c-4657-8db5-ebbfb1c88fb2"}]': finished 2023-12-17T12:04:22.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:22 smithi192 ceph-mon[41899]: osdmap e15: 3 total, 2 up, 3 in 2023-12-17T12:04:22.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:22 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:23.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:23 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/2359083768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:23.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:23 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/2359083768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:23.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:23 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/2359083768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:24.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:24 smithi027 ceph-mon[36871]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:24.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:24 smithi141 ceph-mon[41225]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:24.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:24 smithi192 ceph-mon[41899]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:26.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:26 smithi027 ceph-mon[36871]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:26.422 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:26 smithi192 ceph-mon[41899]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:26.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:26 smithi141 ceph-mon[41225]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:28.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:28 smithi027 ceph-mon[36871]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:28.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:28 smithi141 ceph-mon[41225]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:28.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:28 smithi192 ceph-mon[41899]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:29.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:29 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-17T12:04:29.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:29 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:29.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:29 smithi027 ceph-mon[36871]: Deploying daemon osd.2 on smithi192 2023-12-17T12:04:29.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:29 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-17T12:04:29.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:29 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:29.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:29 smithi141 ceph-mon[41225]: Deploying daemon osd.2 on smithi192 2023-12-17T12:04:29.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:29 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-17T12:04:29.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:29 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:29.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:29 smithi192 ceph-mon[41899]: Deploying daemon osd.2 on smithi192 2023-12-17T12:04:30.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:30 smithi027 ceph-mon[36871]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:30.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:30 smithi192 ceph-mon[41899]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:30.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:30 smithi141 ceph-mon[41225]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:32.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:32 smithi027 ceph-mon[36871]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:32.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:32 smithi141 ceph-mon[41225]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:32.529 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:32 smithi192 ceph-mon[41899]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:34.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:34 smithi192 ceph-mon[41899]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:34.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:34.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:34.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:34.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:34 smithi141 ceph-mon[41225]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:34.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:34.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:34.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:34.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:34 smithi027 ceph-mon[36871]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:34.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:34.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:34.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:36.234 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:36 smithi192 ceph-mon[41899]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:36.234 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.235 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.235 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:36.235 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:36.235 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.298 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.297+0000 7fbfde7fc700 1 -- 172.21.15.192:0/566219958 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbfe802d380 con 0x7fbfd40a13f0 2023-12-17T12:04:36.298 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2023-12-17T12:04:36.301 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbfd40a13f0 msgr2=0x7fbfd40a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:36.301 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbfd40a13f0 0x7fbfd40a38b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b2e0 tx=0x7fbfe400b6b0 comp rx=0 tx=0).stop 2023-12-17T12:04:36.301 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102820 msgr2=0x7fbfe8197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:36.301 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102820 0x7fbfe8197a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800e9d0 tx=0x7fbfd800ece0 comp rx=0 tx=0).stop 2023-12-17T12:04:36.301 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 shutdown_connections 2023-12-17T12:04:36.301 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbfd40a13f0 0x7fbfd40a38b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:36.301 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbfe81031c0 0x7fbfe819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:36.302 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfe8101620 0x7fbfe8197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:36.302 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 --2- 172.21.15.192:0/566219958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102820 0x7fbfe8197a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:36.302 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 >> 172.21.15.192:0/566219958 conn(0x7fbfe80fcbe0 msgr2=0x7fbfe8105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:04:36.302 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 shutdown_connections 2023-12-17T12:04:36.302 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:04:36.299+0000 7fbfef5c6700 1 -- 172.21.15.192:0/566219958 wait complete. 2023-12-17T12:04:36.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:36 smithi027 ceph-mon[36871]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:36.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:36.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:36 smithi141 ceph-mon[41225]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:36.728 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.2.service 2023-12-17T12:04:36.733 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_3... 2023-12-17T12:04:36.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-17T12:04:36.808 INFO:journalctl@ceph.osd.2.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:04:37.163 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:04:37.506 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:37 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:37.506 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:37 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:37.506 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:37 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:37.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:37 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:37.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:37 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:37.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:37 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:37.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:37 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:37.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:37 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:37.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:37 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:37.795 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:37 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-2[47711]: 2023-12-17T12:04:37.504+0000 7fdfd66c0640 -1 osd.2 0 log_to_monitors true 2023-12-17T12:04:38.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:38 smithi027 ceph-mon[36871]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:38.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:38 smithi027 ceph-mon[36871]: from='osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-17T12:04:38.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:38 smithi027 ceph-mon[36871]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-17T12:04:38.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:38 smithi192 ceph-mon[41899]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:38.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:38 smithi192 ceph-mon[41899]: from='osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-17T12:04:38.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:38 smithi192 ceph-mon[41899]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-17T12:04:38.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:38 smithi141 ceph-mon[41225]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:38.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:38 smithi141 ceph-mon[41225]: from='osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-17T12:04:38.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:38 smithi141 ceph-mon[41225]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-17T12:04:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-17T12:04:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-17T12:04:38.911 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2023-12-17T12:04:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2023-12-17T12:04:38.911 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0413455 s, 254 MB/s 2023-12-17T12:04:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2023-12-17T12:04:39.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2023-12-17T12:04:39.537 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-2[47711]: 2023-12-17T12:04:39.299+0000 7fdfcc538700 -1 osd.2 0 waiting for initial osdmap 2023-12-17T12:04:39.537 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-2[47711]: 2023-12-17T12:04:39.304+0000 7fdfc4b2b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:04:39.537 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-mon[41899]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-17T12:04:39.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-mon[41899]: from='osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:04:39.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-mon[41899]: osdmap e16: 3 total, 2 up, 3 in 2023-12-17T12:04:39.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:39.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-mon[41899]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:04:39.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-mon[41899]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:04:39.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-mon[41899]: osdmap e17: 3 total, 2 up, 3 in 2023-12-17T12:04:39.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:39 smithi027 ceph-mon[36871]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-17T12:04:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:39 smithi027 ceph-mon[36871]: from='osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:04:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:39 smithi027 ceph-mon[36871]: osdmap e16: 3 total, 2 up, 3 in 2023-12-17T12:04:39.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:39.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:39 smithi027 ceph-mon[36871]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:04:39.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:39 smithi027 ceph-mon[36871]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:04:39.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:39 smithi027 ceph-mon[36871]: osdmap e17: 3 total, 2 up, 3 in 2023-12-17T12:04:39.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:39.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:39 smithi141 ceph-mon[41225]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-17T12:04:39.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:39 smithi141 ceph-mon[41225]: from='osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:04:39.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:39 smithi141 ceph-mon[41225]: osdmap e16: 3 total, 2 up, 3 in 2023-12-17T12:04:39.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:39.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:39 smithi141 ceph-mon[41225]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:04:39.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:39 smithi141 ceph-mon[41225]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:04:39.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:39 smithi141 ceph-mon[41225]: osdmap e17: 3 total, 2 up, 3 in 2023-12-17T12:04:39.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:39.774 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:04:40.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi192 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi192 to 17271M 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi192 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi192 to 17271M 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:40.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:40.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi192 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi192 to 17271M 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:40.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:40.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:40.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 -- 172.21.15.27:0/2115362676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63b0075600 msgr2=0x7f63b0073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 --2- 172.21.15.27:0/2115362676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63b0075600 0x7f63b0073a60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098f0 tx=0x7f63a4009c00 comp rx=0 tx=0).stop 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 -- 172.21.15.27:0/2115362676 shutdown_connections 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 --2- 172.21.15.27:0/2115362676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63b0075600 0x7f63b0073a60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 --2- 172.21.15.27:0/2115362676 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63b010dc00 0x7f63b010fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 --2- 172.21.15.27:0/2115362676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b0074030 0x7f63b00744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 -- 172.21.15.27:0/2115362676 >> 172.21.15.27:0/2115362676 conn(0x7f63b00fcbe0 msgr2=0x7f63b00ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 -- 172.21.15.27:0/2115362676 shutdown_connections 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.900+0000 7f63b712a700 1 -- 172.21.15.27:0/2115362676 wait complete. 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.901+0000 7f63b712a700 1 Processor -- start 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.901+0000 7f63b712a700 1 -- start start 2023-12-17T12:04:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b712a700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63b0074030 0x7f63b019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b712a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b0075600 0x7f63b019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b712a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63b010dc00 0x7f63b01a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b712a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01a5670 con 0x7f63b0075600 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b712a700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01a57b0 con 0x7f63b0074030 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b712a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01a5940 con 0x7f63b010dc00 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63affff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b0075600 0x7f63b019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63affff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b0075600 0x7f63b019e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57302/0 (socket says 172.21.15.27:57302) 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63affff700 1 -- 172.21.15.27:0/1722871297 learned_addr learned my addr 172.21.15.27:0/1722871297 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b4ec6700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63b0074030 0x7f63b019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b56c7700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63b010dc00 0x7f63b01a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b4ec6700 1 -- 172.21.15.27:0/1722871297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63b010dc00 msgr2=0x7f63b01a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b4ec6700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63b010dc00 0x7f63b01a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b4ec6700 1 -- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b0075600 msgr2=0x7f63b019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b4ec6700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b0075600 0x7f63b019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.902+0000 7f63b4ec6700 1 -- 172.21.15.27:0/1722871297 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a8009d00 con 0x7f63b0074030 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.903+0000 7f63affff700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b0075600 0x7f63b019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.903+0000 7f63b4ec6700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63b0074030 0x7f63b019db80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009230 tx=0x7f63a400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.903+0000 7f63adffb700 1 -- 172.21.15.27:0/1722871297 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a401c070 con 0x7f63b0074030 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.903+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a4009520 con 0x7f63b0074030 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.903+0000 7f63adffb700 1 -- 172.21.15.27:0/1722871297 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63a400bec0 con 0x7f63b0074030 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.903+0000 7f63adffb700 1 -- 172.21.15.27:0/1722871297 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4003900 con 0x7f63b0074030 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.903+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b01a5ea0 con 0x7f63b0074030 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.905+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b0103e60 con 0x7f63b0074030 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.907+0000 7f63adffb700 1 -- 172.21.15.27:0/1722871297 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f63a4020070 con 0x7f63b0074030 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.908+0000 7f63adffb700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63a00a1470 0x7f63a00a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:04:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.908+0000 7f63adffb700 1 -- 172.21.15.27:0/1722871297 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f63a40cc2f0 con 0x7f63b0074030 2023-12-17T12:04:40.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.908+0000 7f63affff700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63a00a1470 0x7f63a00a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:04:40.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.909+0000 7f63affff700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63a00a1470 0x7f63a00a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f639c005950 tx=0x7f639c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:04:40.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:40.913+0000 7f63adffb700 1 -- 172.21.15.27:0/1722871297 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f63a4096f70 con 0x7f63b0074030 2023-12-17T12:04:41.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:41.158+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f63b002d380 con 0x7f63a00a1470 2023-12-17T12:04:41.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:41 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:04:41.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:41 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:04:41.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:41 smithi192 ceph-mon[41899]: osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] boot 2023-12-17T12:04:41.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:41 smithi192 ceph-mon[41899]: osdmap e18: 3 total, 3 up, 3 in 2023-12-17T12:04:41.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:41 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:41.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:41 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:04:41.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:41 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:04:41.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:41 smithi027 ceph-mon[36871]: osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] boot 2023-12-17T12:04:41.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:41 smithi027 ceph-mon[36871]: osdmap e18: 3 total, 3 up, 3 in 2023-12-17T12:04:41.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:41 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:42.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:41 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:04:42.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:41 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:04:42.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:41 smithi141 ceph-mon[41225]: osd.2 [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] boot 2023-12-17T12:04:42.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:41 smithi141 ceph-mon[41225]: osdmap e18: 3 total, 3 up, 3 in 2023-12-17T12:04:42.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:41 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-17T12:04:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:04:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:04:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:04:42.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:04:42.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:42.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-17T12:04:42.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: osdmap e19: 3 total, 3 up, 3 in 2023-12-17T12:04:42.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:04:42.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:42.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:04:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:04:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:04:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:04:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-17T12:04:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: osdmap e19: 3 total, 3 up, 3 in 2023-12-17T12:04:42.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:04:43.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-17T12:04:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:04:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:04:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:04:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:04:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-17T12:04:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: osdmap e19: 3 total, 3 up, 3 in 2023-12-17T12:04:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:04:43.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:43.567+0000 7f63adffb700 1 -- 172.21.15.27:0/1722871297 <== mon.2 v2:172.21.15.141:3300/0 7 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f63a4097340 con 0x7f63b0074030 2023-12-17T12:04:43.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:43 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-17T12:04:43.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:43 smithi192 ceph-mon[41899]: osdmap e20: 3 total, 3 up, 3 in 2023-12-17T12:04:43.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-17T12:04:43.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:43 smithi027 ceph-mon[36871]: osdmap e20: 3 total, 3 up, 3 in 2023-12-17T12:04:43.913 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:04:43 smithi027 sudo[61732]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-17T12:04:43.913 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:04:43 smithi027 sudo[61732]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:43.913 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:04:43 smithi027 sudo[61732]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:43.913 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:04:43 smithi027 sudo[61732]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:44.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:43 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-17T12:04:44.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:43 smithi141 ceph-mon[41225]: osdmap e20: 3 total, 3 up, 3 in 2023-12-17T12:04:44.561 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:04:44 smithi027 sudo[61778]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-17T12:04:44.561 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:04:44 smithi027 sudo[61778]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:44.561 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:04:44 smithi027 sudo[61778]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:44.561 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:04:44 smithi027 sudo[61778]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:44.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:44 smithi192 ceph-mon[41899]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:44.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:44 smithi192 ceph-mon[41899]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-17T12:04:44.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:44 smithi192 ceph-mon[41899]: osdmap e21: 3 total, 3 up, 3 in 2023-12-17T12:04:44.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:44 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1256475954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aed4e22-de72-4ad5-8b0d-0c5d159c7152"}]: dispatch 2023-12-17T12:04:44.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:44 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1256475954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9aed4e22-de72-4ad5-8b0d-0c5d159c7152"}]': finished 2023-12-17T12:04:44.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:44 smithi192 ceph-mon[41899]: osdmap e22: 4 total, 3 up, 4 in 2023-12-17T12:04:44.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:04:44.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:44.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:44 smithi027 ceph-mon[36871]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:44.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:44 smithi027 ceph-mon[36871]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-17T12:04:44.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:44 smithi027 ceph-mon[36871]: osdmap e21: 3 total, 3 up, 3 in 2023-12-17T12:04:44.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:44 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1256475954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aed4e22-de72-4ad5-8b0d-0c5d159c7152"}]: dispatch 2023-12-17T12:04:44.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:44 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1256475954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9aed4e22-de72-4ad5-8b0d-0c5d159c7152"}]': finished 2023-12-17T12:04:44.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:44 smithi027 ceph-mon[36871]: osdmap e22: 4 total, 3 up, 4 in 2023-12-17T12:04:44.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:04:44.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:44 smithi141 ceph-mon[41225]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:44 smithi141 ceph-mon[41225]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-17T12:04:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:44 smithi141 ceph-mon[41225]: osdmap e21: 3 total, 3 up, 3 in 2023-12-17T12:04:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:44 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1256475954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aed4e22-de72-4ad5-8b0d-0c5d159c7152"}]: dispatch 2023-12-17T12:04:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:44 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1256475954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9aed4e22-de72-4ad5-8b0d-0c5d159c7152"}]': finished 2023-12-17T12:04:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:44 smithi141 ceph-mon[41225]: osdmap e22: 4 total, 3 up, 4 in 2023-12-17T12:04:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:04:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:45.016 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:44 smithi141 sudo[50965]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-17T12:04:45.016 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:44 smithi141 sudo[50965]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:45.016 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:44 smithi141 sudo[50965]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:45.016 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:44 smithi141 sudo[50965]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:45.567 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:45 smithi141 sudo[50968]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-17T12:04:45.568 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:45 smithi141 sudo[50968]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:45.568 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:45 smithi141 sudo[50968]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:45.568 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:04:45 smithi141 sudo[50968]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:45.830 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:45 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4129651240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:45.830 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:45 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:45.831 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:45 smithi192 sudo[52803]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-17T12:04:45.831 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:45 smithi192 sudo[52803]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:45.831 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:45 smithi192 sudo[52803]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:45.831 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:45 smithi192 sudo[52803]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:45.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:45 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4129651240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:45.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:45 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:46.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:45 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4129651240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:04:46.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:45 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:46.811 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:46 smithi192 sudo[52806]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-17T12:04:46.812 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:46 smithi192 sudo[52806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:46.812 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:46 smithi192 sudo[52806]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:46.812 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:04:46 smithi192 sudo[52806]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 sudo[62617]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-17T12:04:47.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 sudo[62617]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:47.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 sudo[62617]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:47.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 sudo[62617]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:48.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 sudo[63001]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-17T12:04:48.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 sudo[63001]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:48.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 sudo[63001]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:48.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 sudo[63001]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:48.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 ceph-mon[36871]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:48.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:48.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:47 smithi027 ceph-mon[36871]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:48.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:47 smithi141 ceph-mon[41225]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:48.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:48.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:47 smithi141 ceph-mon[41225]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:48.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:47 smithi192 ceph-mon[41899]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:48.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:48.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:47 smithi192 ceph-mon[41899]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:48 smithi192 sudo[52817]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-17T12:04:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:48 smithi192 sudo[52817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:48 smithi192 sudo[52817]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:48 smithi192 sudo[52817]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:49.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:48 smithi192 sudo[52820]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-17T12:04:49.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:48 smithi192 sudo[52820]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:49.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:48 smithi192 sudo[52820]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:49.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:48 smithi192 sudo[52820]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 sudo[50979]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-17T12:04:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 sudo[50979]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 sudo[50979]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 sudo[50979]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:50.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:49 smithi027 ceph-mon[36871]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:50.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:49 smithi027 ceph-mon[36871]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:50.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:50.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:50.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:50.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:50.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:50.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 sudo[50982]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-17T12:04:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 sudo[50982]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-17T12:04:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 sudo[50982]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-17T12:04:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 sudo[50982]: pam_unix(sudo:session): session closed for user root 2023-12-17T12:04:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 ceph-mon[41225]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 ceph-mon[41225]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:50.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:50.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:50.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:50.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:50.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:49 smithi192 ceph-mon[41899]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:50.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:49 smithi192 ceph-mon[41899]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:50.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:50.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:50.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:50.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:50.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:50.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:51.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:51.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:51.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:51.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:51.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:51.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:51.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:51.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:51.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-17T12:04:51.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:51.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:51.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:51.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:51.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:51.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:51.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:51.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:51.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:51.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-17T12:04:51.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:51.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:51.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:04:51.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:51.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-17T12:04:51.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-17T12:04:51.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-17T12:04:51.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-17T12:04:51.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-17T12:04:51.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-17T12:04:51.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:52.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:51 smithi027 ceph-mon[36871]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-12-17T12:04:52.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:51 smithi027 ceph-mon[36871]: Deploying daemon osd.3 on smithi027 2023-12-17T12:04:52.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:51 smithi141 ceph-mon[41225]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-12-17T12:04:52.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:51 smithi141 ceph-mon[41225]: Deploying daemon osd.3 on smithi027 2023-12-17T12:04:52.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:51 smithi192 ceph-mon[41899]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-12-17T12:04:52.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:51 smithi192 ceph-mon[41899]: Deploying daemon osd.3 on smithi027 2023-12-17T12:04:54.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:53 smithi027 ceph-mon[36871]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-12-17T12:04:54.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:53 smithi141 ceph-mon[41225]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-12-17T12:04:54.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:53 smithi192 ceph-mon[41899]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-12-17T12:04:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:55 smithi027 ceph-mon[36871]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 172 KiB/s wr, 11 op/s 2023-12-17T12:04:56.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:56.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:56.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:55 smithi141 ceph-mon[41225]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 172 KiB/s wr, 11 op/s 2023-12-17T12:04:56.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:56.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:56.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:55 smithi192 ceph-mon[41899]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 172 KiB/s wr, 11 op/s 2023-12-17T12:04:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:04:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:57 smithi141 ceph-mon[41225]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:04:58.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:57 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:57 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:57 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:58.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:57 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:58.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:57 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:57 smithi192 ceph-mon[41899]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:04:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:57 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:57 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:57 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:57 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:57 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:57 smithi027 ceph-mon[36871]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:04:58.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:57 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:57 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:57 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:04:58.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:57 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:04:58.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:57 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:04:58.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.979+0000 7f63adffb700 1 -- 172.21.15.27:0/1722871297 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f63b002d380 con 0x7f63a00a1470 2023-12-17T12:04:58.982 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.981+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63a00a1470 msgr2=0x7f63a00a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.981+0000 7f63b712a700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63a00a1470 0x7f63a00a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f639c005950 tx=0x7f639c009260 comp rx=0 tx=0).stop 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.981+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63b0074030 msgr2=0x7f63b019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.981+0000 7f63b712a700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63b0074030 0x7f63b019db80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009230 tx=0x7f63a400b790 comp rx=0 tx=0).stop 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.982+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 shutdown_connections 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.982+0000 7f63b712a700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63a00a1470 0x7f63a00a3930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.982+0000 7f63b712a700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63b010dc00 0x7f63b01a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.982+0000 7f63b712a700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63b0074030 0x7f63b019db80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.982+0000 7f63b712a700 1 --2- 172.21.15.27:0/1722871297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b0075600 0x7f63b019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.982+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 >> 172.21.15.27:0/1722871297 conn(0x7f63b00fcbe0 msgr2=0x7f63b0100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:04:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.982+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 shutdown_connections 2023-12-17T12:04:58.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:04:58.982+0000 7f63b712a700 1 -- 172.21.15.27:0/1722871297 wait complete. 2023-12-17T12:04:59.375 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.3.service 2023-12-17T12:04:59.380 INFO:tasks.cephadm:Deploying osd.4 on smithi141 with /dev/vg_nvme/lv_3... 2023-12-17T12:04:59.380 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-17T12:04:59.460 INFO:journalctl@ceph.osd.3.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:04:59.789 INFO:teuthology.orchestra.run.smithi141.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.b/config 2023-12-17T12:05:00.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:59 smithi141 ceph-mon[41225]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:05:00.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:00.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:00.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:04:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:00.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:59 smithi192 ceph-mon[41899]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:05:00.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:00.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:00.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:04:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:00.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:59 smithi027 ceph-mon[36871]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:05:00.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:00.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:00.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:04:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:00.413 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:04:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3[63997]: 2023-12-17T12:04:59.991+0000 7f542c405640 -1 osd.3 0 log_to_monitors true 2023-12-17T12:05:01.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:00 smithi141 ceph-mon[41225]: from='osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-17T12:05:01.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:00 smithi141 ceph-mon[41225]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-17T12:05:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:00 smithi192 ceph-mon[41899]: from='osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-17T12:05:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:00 smithi192 ceph-mon[41899]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-17T12:05:01.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:00 smithi027 ceph-mon[36871]: from='osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-17T12:05:01.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:00 smithi027 ceph-mon[36871]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-17T12:05:01.519 INFO:teuthology.orchestra.run.smithi141.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-17T12:05:01.519 INFO:teuthology.orchestra.run.smithi141.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-17T12:05:01.519 INFO:teuthology.orchestra.run.smithi141.stderr: stderr: 10+0 records in 2023-12-17T12:05:01.519 INFO:teuthology.orchestra.run.smithi141.stderr:10+0 records out 2023-12-17T12:05:01.519 INFO:teuthology.orchestra.run.smithi141.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.042993 s, 244 MB/s 2023-12-17T12:05:01.519 INFO:teuthology.orchestra.run.smithi141.stderr:--> Zapping successful for: 2023-12-17T12:05:01.912 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi141:vg_nvme/lv_3 2023-12-17T12:05:02.264 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3[63997]: 2023-12-17T12:05:02.003+0000 7f5421a7c700 -1 osd.3 0 waiting for initial osdmap 2023-12-17T12:05:02.264 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3[63997]: 2023-12-17T12:05:02.012+0000 7f541b872700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:05:02.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:01 smithi027 ceph-mon[36871]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:01 smithi027 ceph-mon[36871]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:01 smithi027 ceph-mon[36871]: osdmap e23: 4 total, 3 up, 4 in 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:01 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:01 smithi027 ceph-mon[36871]: from='osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:01 smithi027 ceph-mon[36871]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:01 smithi141 ceph-mon[41225]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:01 smithi141 ceph-mon[41225]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:01 smithi141 ceph-mon[41225]: osdmap e23: 4 total, 3 up, 4 in 2023-12-17T12:05:02.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:01 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:02.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:01 smithi141 ceph-mon[41225]: from='osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:05:02.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:01 smithi141 ceph-mon[41225]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:05:02.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:01 smithi192 ceph-mon[41899]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-17T12:05:02.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:01 smithi192 ceph-mon[41899]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-17T12:05:02.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:01 smithi192 ceph-mon[41899]: osdmap e23: 4 total, 3 up, 4 in 2023-12-17T12:05:02.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:01 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:02.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:01 smithi192 ceph-mon[41899]: from='osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:05:02.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:01 smithi192 ceph-mon[41899]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:05:02.318 INFO:teuthology.orchestra.run.smithi141.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.b/config 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: osdmap e24: 4 total, 3 up, 4 in 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi027 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi027 to 8635M 2023-12-17T12:05:03.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:03.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:03.266 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:05:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: osdmap e24: 4 total, 3 up, 4 in 2023-12-17T12:05:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi027 2023-12-17T12:05:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi027 to 8635M 2023-12-17T12:05:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:05:03.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: osdmap e24: 4 total, 3 up, 4 in 2023-12-17T12:05:03.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:03.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi027 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi027 to 8635M 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:03.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:03.569 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.567+0000 7f667ed28700 1 -- 172.21.15.141:0/4225991701 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66781031c0 msgr2=0x7f6678109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:03.569 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.567+0000 7f667ed28700 1 --2- 172.21.15.141:0/4225991701 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66781031c0 0x7f6678109a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f667000b0d0 tx=0x7f667000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:05:03.569 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.567+0000 7f667ed28700 1 -- 172.21.15.141:0/4225991701 shutdown_connections 2023-12-17T12:05:03.569 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.568+0000 7f667ed28700 1 --2- 172.21.15.141:0/4225991701 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66781031c0 0x7f6678109a70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:03.570 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.568+0000 7f667ed28700 1 --2- 172.21.15.141:0/4225991701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6678101620 0x7f6678101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:03.570 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.568+0000 7f667ed28700 1 --2- 172.21.15.141:0/4225991701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6678102820 0x7f6678102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:03.570 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.568+0000 7f667ed28700 1 -- 172.21.15.141:0/4225991701 >> 172.21.15.141:0/4225991701 conn(0x7f66780fcbe0 msgr2=0x7f66780ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:05:03.570 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.568+0000 7f667ed28700 1 -- 172.21.15.141:0/4225991701 shutdown_connections 2023-12-17T12:05:03.570 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.568+0000 7f667ed28700 1 -- 172.21.15.141:0/4225991701 wait complete. 2023-12-17T12:05:03.571 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667ed28700 1 Processor -- start 2023-12-17T12:05:03.571 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667ed28700 1 -- start start 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667ed28700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6678101620 0x7f66781975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667ed28700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6678102820 0x7f6678197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667ed28700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66781031c0 0x7f667819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667ed28700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667819f0e0 con 0x7f66781031c0 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667ed28700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667819f220 con 0x7f6678101620 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667ed28700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667819f3b0 con 0x7f6678102820 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667cac4700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6678101620 0x7f66781975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667cac4700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6678101620 0x7f66781975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:36198/0 (socket says 172.21.15.141:36198) 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.569+0000 7f667cac4700 1 -- 172.21.15.141:0/1584749235 learned_addr learned my addr 172.21.15.141:0/1584749235 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f6677fff700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6678102820 0x7f6678197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f667d2c5700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66781031c0 0x7f667819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f667cac4700 1 -- 172.21.15.141:0/1584749235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6678102820 msgr2=0x7f6678197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f667cac4700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6678102820 0x7f6678197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:03.572 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f667cac4700 1 -- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66781031c0 msgr2=0x7f667819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:03.573 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f667cac4700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66781031c0 0x7f667819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:03.573 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f667cac4700 1 -- 172.21.15.141:0/1584749235 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670009d00 con 0x7f6678101620 2023-12-17T12:05:03.573 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f667cac4700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6678101620 0x7f66781975a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f666c00d980 tx=0x7f666c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:05:03.573 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f6675ffb700 1 -- 172.21.15.141:0/1584749235 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c004020 con 0x7f6678101620 2023-12-17T12:05:03.573 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f6675ffb700 1 -- 172.21.15.141:0/1584749235 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f666c00f690 con 0x7f6678101620 2023-12-17T12:05:03.573 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667819f690 con 0x7f6678101620 2023-12-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.570+0000 7f6675ffb700 1 -- 172.21.15.141:0/1584749235 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c003a00 con 0x7f6678101620 2023-12-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.571+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667819fbe0 con 0x7f6678101620 2023-12-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.572+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6678108ad0 con 0x7f6678101620 2023-12-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.573+0000 7f6675ffb700 1 -- 172.21.15.141:0/1584749235 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f666c00f800 con 0x7f6678101620 2023-12-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.575+0000 7f6675ffb700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66680a1500 0x7f66680a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.575+0000 7f6675ffb700 1 -- 172.21.15.141:0/1584749235 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3702+0+0 (secure 0 0 0) 0x7f666c0cb6b0 con 0x7f6678101620 2023-12-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.575+0000 7f6677fff700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66680a1500 0x7f66680a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.576+0000 7f6677fff700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66680a1500 0x7f66680a39c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6664007be0 tx=0x7f66640073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:05:03.581 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.580+0000 7f6675ffb700 1 -- 172.21.15.141:0/1584749235 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f666c095e70 con 0x7f6678101620 2023-12-17T12:05:03.789 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:03.788+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f667802d380 con 0x7f66680a1500 2023-12-17T12:05:04.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:05:04.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:05:04.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:05:04.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] boot 2023-12-17T12:05:04.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: osdmap e25: 4 total, 4 up, 4 in 2023-12-17T12:05:04.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:04.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:04.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:04.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:04.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:04 smithi141 ceph-mon[41225]: osdmap e26: 4 total, 4 up, 4 in 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] boot 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: osdmap e25: 4 total, 4 up, 4 in 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:04.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:04 smithi192 ceph-mon[41899]: osdmap e26: 4 total, 4 up, 4 in 2023-12-17T12:05:04.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:05:04.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:05:04.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-17T12:05:04.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: osd.3 [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] boot 2023-12-17T12:05:04.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: osdmap e25: 4 total, 4 up, 4 in 2023-12-17T12:05:04.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-17T12:05:04.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:04.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:04.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:04.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:04 smithi027 ceph-mon[36871]: osdmap e26: 4 total, 4 up, 4 in 2023-12-17T12:05:05.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:05 smithi141 ceph-mon[41225]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:05.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:05 smithi141 ceph-mon[41225]: osdmap e27: 4 total, 4 up, 4 in 2023-12-17T12:05:05.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:05 smithi192 ceph-mon[41899]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:05.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:05 smithi192 ceph-mon[41899]: osdmap e27: 4 total, 4 up, 4 in 2023-12-17T12:05:05.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:05 smithi027 ceph-mon[36871]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:05.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:05 smithi027 ceph-mon[36871]: osdmap e27: 4 total, 4 up, 4 in 2023-12-17T12:05:06.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:06 smithi192 ceph-mon[41899]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:06.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:06 smithi027 ceph-mon[36871]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:06.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:06 smithi141 ceph-mon[41225]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:07.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:07 smithi192 ceph-mon[41899]: osdmap e28: 4 total, 4 up, 4 in 2023-12-17T12:05:07.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:07 smithi192 ceph-mon[41899]: from='client.? 172.21.15.141:0/37296929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]: dispatch 2023-12-17T12:05:07.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:07 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]: dispatch 2023-12-17T12:05:07.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:07 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]': finished 2023-12-17T12:05:07.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:07 smithi192 ceph-mon[41899]: osdmap e29: 5 total, 4 up, 5 in 2023-12-17T12:05:07.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:07 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:07.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:07 smithi027 ceph-mon[36871]: osdmap e28: 4 total, 4 up, 4 in 2023-12-17T12:05:07.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:07 smithi027 ceph-mon[36871]: from='client.? 172.21.15.141:0/37296929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]: dispatch 2023-12-17T12:05:07.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:07 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]: dispatch 2023-12-17T12:05:07.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:07 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]': finished 2023-12-17T12:05:07.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:07 smithi027 ceph-mon[36871]: osdmap e29: 5 total, 4 up, 5 in 2023-12-17T12:05:07.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:07 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:07.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:07 smithi141 ceph-mon[41225]: osdmap e28: 4 total, 4 up, 4 in 2023-12-17T12:05:07.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:07 smithi141 ceph-mon[41225]: from='client.? 172.21.15.141:0/37296929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]: dispatch 2023-12-17T12:05:07.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:07 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]: dispatch 2023-12-17T12:05:07.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:07 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf"}]': finished 2023-12-17T12:05:07.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:07 smithi141 ceph-mon[41225]: osdmap e29: 5 total, 4 up, 5 in 2023-12-17T12:05:07.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:07 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:08.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:08 smithi141 ceph-mon[41225]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:08.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:08 smithi141 ceph-mon[41225]: from='client.? 172.21.15.141:0/1914756066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:08.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:08 smithi192 ceph-mon[41899]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:08.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:08 smithi192 ceph-mon[41899]: from='client.? 172.21.15.141:0/1914756066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:08.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:08 smithi027 ceph-mon[36871]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:08.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:08 smithi027 ceph-mon[36871]: from='client.? 172.21.15.141:0/1914756066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:10.499 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:10 smithi141 ceph-mon[41225]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:10.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:10 smithi192 ceph-mon[41899]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:10.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:10 smithi027 ceph-mon[36871]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:12.402 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:12 smithi141 ceph-mon[41225]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:05:12.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:12 smithi192 ceph-mon[41899]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:05:12.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:12 smithi027 ceph-mon[36871]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:05:14.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:14 smithi141 ceph-mon[41225]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-12-17T12:05:14.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-17T12:05:14.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:14.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:14 smithi141 ceph-mon[41225]: Deploying daemon osd.4 on smithi141 2023-12-17T12:05:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:14 smithi192 ceph-mon[41899]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-12-17T12:05:14.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-17T12:05:14.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:14.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:14 smithi192 ceph-mon[41899]: Deploying daemon osd.4 on smithi141 2023-12-17T12:05:14.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:14 smithi027 ceph-mon[36871]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-12-17T12:05:14.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-17T12:05:14.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:14.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:14 smithi027 ceph-mon[36871]: Deploying daemon osd.4 on smithi141 2023-12-17T12:05:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:16 smithi141 ceph-mon[41225]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-17T12:05:16.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:16 smithi192 ceph-mon[41899]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-17T12:05:16.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:16 smithi027 ceph-mon[36871]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-17T12:05:18.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:18 smithi141 ceph-mon[41225]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-12-17T12:05:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:18 smithi192 ceph-mon[41899]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-12-17T12:05:18.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:18 smithi027 ceph-mon[36871]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-12-17T12:05:19.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:19.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:19.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:19.653 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:19.653 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:19.654 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:19.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:19.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:19.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:20.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:20 smithi192 ceph-mon[41899]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:05:20.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:20.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:20.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:20.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:20.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:20 smithi027 ceph-mon[36871]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:05:20.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:20.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:20.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:20.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:20.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:20.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:20 smithi141 ceph-mon[41225]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:05:20.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:20 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:20.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:20 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:20.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:20 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:20.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:20 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:20.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:20 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:21.207 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.205+0000 7f6675ffb700 1 -- 172.21.15.141:0/1584749235 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f667802d380 con 0x7f66680a1500 2023-12-17T12:05:21.207 INFO:teuthology.orchestra.run.smithi141.stdout:Created osd(s) 4 on host 'smithi141' 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66680a1500 msgr2=0x7f66680a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66680a1500 0x7f66680a39c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6664007be0 tx=0x7f66640073d0 comp rx=0 tx=0).stop 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6678101620 msgr2=0x7f66781975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6678101620 0x7f66781975a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f666c00d980 tx=0x7f666c00dc90 comp rx=0 tx=0).stop 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 shutdown_connections 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66680a1500 0x7f66680a39c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66781031c0 0x7f667819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6678101620 0x7f66781975a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 --2- 172.21.15.141:0/1584749235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6678102820 0x7f6678197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 >> 172.21.15.141:0/1584749235 conn(0x7f66780fcbe0 msgr2=0x7f6678105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:05:21.210 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.208+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 shutdown_connections 2023-12-17T12:05:21.211 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:05:21.209+0000 7f667ed28700 1 -- 172.21.15.141:0/1584749235 wait complete. 2023-12-17T12:05:21.616 DEBUG:teuthology.orchestra.run.smithi141:osd.4> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.4.service 2023-12-17T12:05:21.621 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2023-12-17T12:05:21.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-17T12:05:21.699 INFO:journalctl@ceph.osd.4.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:05:22.032 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:05:22.398 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:22 smithi141 ceph-mon[41225]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:05:22.398 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:22.399 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:22.399 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:22 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:22.399 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:05:22 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-4[54156]: 2023-12-17T12:05:22.306+0000 7f7aef0c4640 -1 osd.4 0 log_to_monitors true 2023-12-17T12:05:22.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:22 smithi192 ceph-mon[41899]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:05:22.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:22 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:22.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:22 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:22.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:22 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:22.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:22 smithi027 ceph-mon[36871]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:05:22.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:22.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:22.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:22 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:23.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:23 smithi192 ceph-mon[41899]: from='osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-17T12:05:23.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:23 smithi192 ceph-mon[41899]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-17T12:05:23.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:23 smithi141 ceph-mon[41225]: from='osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-17T12:05:23.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:23 smithi141 ceph-mon[41225]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-17T12:05:23.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:23 smithi027 ceph-mon[36871]: from='osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-17T12:05:23.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:23 smithi027 ceph-mon[36871]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-17T12:05:23.855 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-17T12:05:23.855 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-17T12:05:23.855 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-12-17T12:05:23.855 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-12-17T12:05:23.855 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0304266 s, 345 MB/s 2023-12-17T12:05:23.855 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-12-17T12:05:24.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2023-12-17T12:05:24.446 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:05:24 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-4[54156]: 2023-12-17T12:05:24.218+0000 7f7ae473b700 -1 osd.4 0 waiting for initial osdmap 2023-12-17T12:05:24.447 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:05:24 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-4[54156]: 2023-12-17T12:05:24.225+0000 7f7adfd34700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:05:24.447 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:24 smithi141 ceph-mon[41225]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:24.447 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:24 smithi141 ceph-mon[41225]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-17T12:05:24.447 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:24 smithi141 ceph-mon[41225]: from='osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:05:24.447 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:24 smithi141 ceph-mon[41225]: osdmap e30: 5 total, 4 up, 5 in 2023-12-17T12:05:24.447 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:24.447 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:24 smithi141 ceph-mon[41225]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:05:24.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:24 smithi192 ceph-mon[41899]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:24.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:24 smithi192 ceph-mon[41899]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-17T12:05:24.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:24 smithi192 ceph-mon[41899]: from='osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:05:24.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:24 smithi192 ceph-mon[41899]: osdmap e30: 5 total, 4 up, 5 in 2023-12-17T12:05:24.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:24.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:24 smithi192 ceph-mon[41899]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:05:24.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:24 smithi027 ceph-mon[36871]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:24.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:24 smithi027 ceph-mon[36871]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-17T12:05:24.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:24 smithi027 ceph-mon[36871]: from='osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:05:24.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:24 smithi027 ceph-mon[36871]: osdmap e30: 5 total, 4 up, 5 in 2023-12-17T12:05:24.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:24.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:24 smithi027 ceph-mon[36871]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:05:24.699 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:05:25.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: osdmap e31: 5 total, 4 up, 5 in 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi141 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi141 to 8635M 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:25.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:25.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:25 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:05:25.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: osdmap e31: 5 total, 4 up, 5 in 2023-12-17T12:05:25.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:25.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:25.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi141 2023-12-17T12:05:25.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:25.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:25.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi141 to 8635M 2023-12-17T12:05:25.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:25.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:25.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:25 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:05:25.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: osdmap e31: 5 total, 4 up, 5 in 2023-12-17T12:05:25.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:25.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:25.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi141 2023-12-17T12:05:25.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:25.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:25.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi141 to 8635M 2023-12-17T12:05:25.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:25.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:25.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:25 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:25.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 -- 172.21.15.192:0/1085436978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c01031c0 msgr2=0x7f97c0109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:25.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 --2- 172.21.15.192:0/1085436978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c01031c0 0x7f97c0109a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f97b800b0a0 tx=0x7f97b800b3b0 comp rx=0 tx=0).stop 2023-12-17T12:05:25.917 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 -- 172.21.15.192:0/1085436978 shutdown_connections 2023-12-17T12:05:25.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 --2- 172.21.15.192:0/1085436978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c01031c0 0x7f97c0109a70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:25.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 --2- 172.21.15.192:0/1085436978 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f97c0101620 0x7f97c0101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:25.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 --2- 172.21.15.192:0/1085436978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c0102820 0x7f97c0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:25.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 -- 172.21.15.192:0/1085436978 >> 172.21.15.192:0/1085436978 conn(0x7f97c00fcbe0 msgr2=0x7f97c00ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:05:25.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 -- 172.21.15.192:0/1085436978 shutdown_connections 2023-12-17T12:05:25.918 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.916+0000 7f97c62d6700 1 -- 172.21.15.192:0/1085436978 wait complete. 2023-12-17T12:05:25.919 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.917+0000 7f97c62d6700 1 Processor -- start 2023-12-17T12:05:25.919 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.917+0000 7f97c62d6700 1 -- start start 2023-12-17T12:05:25.919 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.917+0000 7f97c62d6700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f97c0101620 0x7f97c0197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.917+0000 7f97c62d6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c0102820 0x7f97c0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.917+0000 7f97c62d6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c01031c0 0x7f97c019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.917+0000 7f97c62d6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019f0d0 con 0x7f97c01031c0 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.917+0000 7f97c62d6700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019f210 con 0x7f97c0101620 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.917+0000 7f97c62d6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c019f3a0 con 0x7f97c0102820 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c0102820 0x7f97c0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c0102820 0x7f97c0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:36688/0 (socket says 172.21.15.192:36688) 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 -- 172.21.15.192:0/4236733013 learned_addr learned my addr 172.21.15.192:0/4236733013 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97c4873700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c01031c0 0x7f97c019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bffff700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f97c0101620 0x7f97c0197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 -- 172.21.15.192:0/4236733013 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f97c0101620 msgr2=0x7f97c0197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f97c0101620 0x7f97c0197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 -- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c01031c0 msgr2=0x7f97c019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c01031c0 0x7f97c019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:25.920 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 -- 172.21.15.192:0/4236733013 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8009d00 con 0x7f97c0102820 2023-12-17T12:05:25.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97c4873700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c01031c0 0x7f97c019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:05:25.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.918+0000 7f97bf7fe700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c0102820 0x7f97c0197ad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f97b400d980 tx=0x7f97b400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:05:25.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.919+0000 7f97bd7fa700 1 -- 172.21.15.192:0/4236733013 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4009b40 con 0x7f97c0102820 2023-12-17T12:05:25.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.919+0000 7f97bd7fa700 1 -- 172.21.15.192:0/4236733013 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97b400b6e0 con 0x7f97c0102820 2023-12-17T12:05:25.921 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.919+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c019f680 con 0x7f97c0102820 2023-12-17T12:05:25.922 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.919+0000 7f97bd7fa700 1 -- 172.21.15.192:0/4236733013 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b400f620 con 0x7f97c0102820 2023-12-17T12:05:25.922 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.919+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c019fb80 con 0x7f97c0102820 2023-12-17T12:05:25.922 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.920+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c0108aa0 con 0x7f97c0102820 2023-12-17T12:05:25.923 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.922+0000 7f97bd7fa700 1 -- 172.21.15.192:0/4236733013 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f97b4010460 con 0x7f97c0102820 2023-12-17T12:05:25.925 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.924+0000 7f97bd7fa700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f97a80a1480 0x7f97a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:25.925 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.924+0000 7f97bd7fa700 1 -- 172.21.15.192:0/4236733013 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f97b40cbc30 con 0x7f97c0102820 2023-12-17T12:05:25.926 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.925+0000 7f97bffff700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f97a80a1480 0x7f97a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:25.927 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.925+0000 7f97bffff700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f97a80a1480 0x7f97a80a3940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97b0005950 tx=0x7f97b00058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:05:25.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:25.928+0000 7f97bd7fa700 1 -- 172.21.15.192:0/4236733013 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f97b4096260 con 0x7f97c0102820 2023-12-17T12:05:26.134 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:26.133+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f97c002d380 con 0x7f97a80a1480 2023-12-17T12:05:26.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] boot 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: osdmap e32: 5 total, 5 up, 5 in 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:26.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:26.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:05:26.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:05:26.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:26.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:26.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] boot 2023-12-17T12:05:26.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: osdmap e32: 5 total, 5 up, 5 in 2023-12-17T12:05:26.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:26.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:26.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:26.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:05:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:05:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-17T12:05:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: osd.4 [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] boot 2023-12-17T12:05:26.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: osdmap e32: 5 total, 5 up, 5 in 2023-12-17T12:05:26.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-17T12:05:26.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:26.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:26.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:27.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:27 smithi141 ceph-mon[41225]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:27.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:27 smithi141 ceph-mon[41225]: osdmap e33: 5 total, 5 up, 5 in 2023-12-17T12:05:27.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:27 smithi192 ceph-mon[41899]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:27.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:27 smithi192 ceph-mon[41899]: osdmap e33: 5 total, 5 up, 5 in 2023-12-17T12:05:27.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:27 smithi027 ceph-mon[36871]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:27.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:27 smithi027 ceph-mon[36871]: osdmap e33: 5 total, 5 up, 5 in 2023-12-17T12:05:28.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:28 smithi141 ceph-mon[41225]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:28.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:28 smithi192 ceph-mon[41899]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:28 smithi027 ceph-mon[36871]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:29.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:29 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/3941146744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]: dispatch 2023-12-17T12:05:29.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:29 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]: dispatch 2023-12-17T12:05:29.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:29 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]': finished 2023-12-17T12:05:29.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:29 smithi141 ceph-mon[41225]: osdmap e34: 6 total, 5 up, 6 in 2023-12-17T12:05:29.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:29 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:29.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:29 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/3941146744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]: dispatch 2023-12-17T12:05:29.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:29 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]: dispatch 2023-12-17T12:05:29.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:29 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]': finished 2023-12-17T12:05:29.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:29 smithi192 ceph-mon[41899]: osdmap e34: 6 total, 5 up, 6 in 2023-12-17T12:05:29.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:29 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:29 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/3941146744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]: dispatch 2023-12-17T12:05:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:29 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]: dispatch 2023-12-17T12:05:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:29 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee59d302-1490-4e7e-ab79-af6243c4e97b"}]': finished 2023-12-17T12:05:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:29 smithi027 ceph-mon[36871]: osdmap e34: 6 total, 5 up, 6 in 2023-12-17T12:05:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:29 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:30.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:30 smithi141 ceph-mon[41225]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:30.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:30 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/2667376531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:30.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:30 smithi192 ceph-mon[41899]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:30.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:30 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/2667376531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:30.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:30 smithi027 ceph-mon[36871]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:30.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:30 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/2667376531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:32.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:32 smithi141 ceph-mon[41225]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:32.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:32 smithi192 ceph-mon[41899]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:32.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:32 smithi027 ceph-mon[36871]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:34.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:34 smithi141 ceph-mon[41225]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:34.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:34 smithi192 ceph-mon[41899]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:34.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:34 smithi027 ceph-mon[36871]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:36 smithi141 ceph-mon[41225]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-17T12:05:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:36.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:36 smithi141 ceph-mon[41225]: Deploying daemon osd.5 on smithi192 2023-12-17T12:05:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:36 smithi192 ceph-mon[41899]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-17T12:05:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:36 smithi192 ceph-mon[41899]: Deploying daemon osd.5 on smithi192 2023-12-17T12:05:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:36 smithi027 ceph-mon[36871]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-17T12:05:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:36 smithi027 ceph-mon[36871]: Deploying daemon osd.5 on smithi192 2023-12-17T12:05:38.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:38 smithi192 ceph-mon[41899]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:38.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:38 smithi141 ceph-mon[41225]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:38.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:38 smithi027 ceph-mon[36871]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:40.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:40 smithi141 ceph-mon[41225]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:40.519 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:40 smithi192 ceph-mon[41899]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:40.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:40 smithi027 ceph-mon[36871]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:42.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:41 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:42.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:41 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:42.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:41 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:41 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:41 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:41 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:41 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:41 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:41 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:42 smithi192 ceph-mon[41899]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:42.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:43.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:42 smithi141 ceph-mon[41225]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:43.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:43.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:42 smithi027 ceph-mon[36871]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:43.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:43.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:43.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:43.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:43.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:43.754 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.752+0000 7f97bd7fa700 1 -- 172.21.15.192:0/4236733013 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f97c002d380 con 0x7f97a80a1480 2023-12-17T12:05:43.754 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2023-12-17T12:05:43.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f97a80a1480 msgr2=0x7f97a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:43.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f97a80a1480 0x7f97a80a3940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97b0005950 tx=0x7f97b00058e0 comp rx=0 tx=0).stop 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c0102820 msgr2=0x7f97c0197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c0102820 0x7f97c0197ad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f97b400d980 tx=0x7f97b400dd40 comp rx=0 tx=0).stop 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 shutdown_connections 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f97a80a1480 0x7f97a80a3940 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c01031c0 0x7f97c019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f97c0101620 0x7f97c0197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 --2- 172.21.15.192:0/4236733013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97c0102820 0x7f97c0197ad0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 >> 172.21.15.192:0/4236733013 conn(0x7f97c00fcbe0 msgr2=0x7f97c0105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 shutdown_connections 2023-12-17T12:05:43.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:05:43.755+0000 7f97c62d6700 1 -- 172.21.15.192:0/4236733013 wait complete. 2023-12-17T12:05:44.204 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.5.service 2023-12-17T12:05:44.209 INFO:tasks.cephadm:Deploying osd.6 on smithi027 with /dev/vg_nvme/lv_2... 2023-12-17T12:05:44.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-17T12:05:44.287 INFO:journalctl@ceph.osd.5.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:05:44.639 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:05:44.861 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:44 smithi192 ceph-mon[41899]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:44.861 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:44.861 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:44.861 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:44.862 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:05:44 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:05:44.719+0000 7f8c1997b640 -1 osd.5 0 log_to_monitors true 2023-12-17T12:05:44.973 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:44 smithi027 ceph-mon[36871]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:44.973 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:44.973 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:44.973 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:44 smithi141 ceph-mon[41225]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:45.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:05:45.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:45 smithi027 ceph-mon[36871]: from='osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-17T12:05:45.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:45 smithi027 ceph-mon[36871]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-17T12:05:46.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:45 smithi141 ceph-mon[41225]: from='osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-17T12:05:46.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:45 smithi141 ceph-mon[41225]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-17T12:05:46.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:45 smithi192 ceph-mon[41899]: from='osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-17T12:05:46.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:45 smithi192 ceph-mon[41899]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-17T12:05:46.485 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-17T12:05:46.485 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-17T12:05:46.485 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2023-12-17T12:05:46.485 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2023-12-17T12:05:46.485 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0309184 s, 339 MB/s 2023-12-17T12:05:46.486 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2023-12-17T12:05:46.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:46 smithi027 ceph-mon[36871]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:46.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:46 smithi027 ceph-mon[36871]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-17T12:05:46.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:46 smithi027 ceph-mon[36871]: from='osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:05:46.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:46 smithi027 ceph-mon[36871]: osdmap e35: 6 total, 5 up, 6 in 2023-12-17T12:05:46.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:46 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:46.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:46 smithi027 ceph-mon[36871]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:05:46.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2023-12-17T12:05:46.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:46 smithi192 ceph-mon[41899]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:46.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:46 smithi192 ceph-mon[41899]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-17T12:05:46.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:46 smithi192 ceph-mon[41899]: from='osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:05:46.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:46 smithi192 ceph-mon[41899]: osdmap e35: 6 total, 5 up, 6 in 2023-12-17T12:05:46.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:46 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:46.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:46 smithi192 ceph-mon[41899]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:05:46.969 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:05:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:05:46.708+0000 7f8c0eff2700 -1 osd.5 0 waiting for initial osdmap 2023-12-17T12:05:46.969 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:05:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:05:46.729+0000 7f8c09dea700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:05:47.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:46 smithi141 ceph-mon[41225]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:47.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:46 smithi141 ceph-mon[41225]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-17T12:05:47.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:46 smithi141 ceph-mon[41225]: from='osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:05:47.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:46 smithi141 ceph-mon[41225]: osdmap e35: 6 total, 5 up, 6 in 2023-12-17T12:05:47.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:46 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:47.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:46 smithi141 ceph-mon[41225]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:05:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: osdmap e36: 6 total, 5 up, 6 in 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:47.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:47.893 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:47.893 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:05:48.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: osdmap e36: 6 total, 5 up, 6 in 2023-12-17T12:05:48.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:48.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:05:48.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: osdmap e36: 6 total, 5 up, 6 in 2023-12-17T12:05:48.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:05:48.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:05:48.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 -- 172.21.15.27:0/3355761064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 msgr2=0x7f36dc109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:48.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3355761064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 0x7f36dc109a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009fc0 tx=0x7f36d400b330 comp rx=0 tx=0).stop 2023-12-17T12:05:48.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 -- 172.21.15.27:0/3355761064 shutdown_connections 2023-12-17T12:05:48.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3355761064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc101620 0x7f36dc101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:48.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3355761064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 0x7f36dc109a70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:48.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3355761064 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f36dc102820 0x7f36dc102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:48.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 -- 172.21.15.27:0/3355761064 >> 172.21.15.27:0/3355761064 conn(0x7f36dc0fcbe0 msgr2=0x7f36dc0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:05:48.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 -- 172.21.15.27:0/3355761064 shutdown_connections 2023-12-17T12:05:48.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.556+0000 7f36e44a3700 1 -- 172.21.15.27:0/3355761064 wait complete. 2023-12-17T12:05:48.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.557+0000 7f36e44a3700 1 Processor -- start 2023-12-17T12:05:48.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.557+0000 7f36e44a3700 1 -- start start 2023-12-17T12:05:48.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e44a3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc101620 0x7f36dc107360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:48.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e44a3700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f36dc102820 0x7f36dc1078a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e44a3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 0x7f36dc103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e44a3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc104680 con 0x7f36dc1031c0 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e44a3700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc1047f0 con 0x7f36dc102820 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e44a3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc1049b0 con 0x7f36dc101620 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e2a40700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 0x7f36dc103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e2a40700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 0x7f36dc103ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39674/0 (socket says 172.21.15.27:39674) 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e2a40700 1 -- 172.21.15.27:0/3091593879 learned_addr learned my addr 172.21.15.27:0/3091593879 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e223f700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc101620 0x7f36dc107360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.558+0000 7f36e1a3e700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f36dc102820 0x7f36dc1078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e2a40700 1 -- 172.21.15.27:0/3091593879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc101620 msgr2=0x7f36dc107360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e2a40700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc101620 0x7f36dc107360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:48.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e2a40700 1 -- 172.21.15.27:0/3091593879 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f36dc102820 msgr2=0x7f36dc1078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:05:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e2a40700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f36dc102820 0x7f36dc1078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:05:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e2a40700 1 -- 172.21.15.27:0/3091593879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d80095f0 con 0x7f36dc1031c0 2023-12-17T12:05:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e1a3e700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f36dc102820 0x7f36dc1078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:05:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e2a40700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 0x7f36dc103ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f36d4008c80 tx=0x7f36d4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:05:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36d37fe700 1 -- 172.21.15.27:0/3091593879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d400e040 con 0x7f36dc1031c0 2023-12-17T12:05:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36d37fe700 1 -- 172.21.15.27:0/3091593879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36d400bc00 con 0x7f36dc1031c0 2023-12-17T12:05:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36d37fe700 1 -- 172.21.15.27:0/3091593879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4005040 con 0x7f36dc1031c0 2023-12-17T12:05:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d4009c40 con 0x7f36dc1031c0 2023-12-17T12:05:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.559+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc1a5ee0 con 0x7f36dc1031c0 2023-12-17T12:05:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.560+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc04f290 con 0x7f36dc1031c0 2023-12-17T12:05:48.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.563+0000 7f36d37fe700 1 -- 172.21.15.27:0/3091593879 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f36d401d470 con 0x7f36dc1031c0 2023-12-17T12:05:48.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.564+0000 7f36d37fe700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36c80a1500 0x7f36c80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:05:48.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.564+0000 7f36d37fe700 1 -- 172.21.15.27:0/3091593879 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f36d40cd660 con 0x7f36dc1031c0 2023-12-17T12:05:48.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.564+0000 7f36e223f700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36c80a1500 0x7f36c80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:05:48.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.565+0000 7f36e223f700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36c80a1500 0x7f36c80a39c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36dc102680 tx=0x7f36d8009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:05:48.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.567+0000 7f36d37fe700 1 -- 172.21.15.27:0/3091593879 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f36d4097b50 con 0x7f36dc1031c0 2023-12-17T12:05:48.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:05:48.788+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f36dc074f20 con 0x7f36c80a1500 2023-12-17T12:05:49.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:05:49.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:05:49.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:49.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi192 2023-12-17T12:05:49.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi192 to 8635M 2023-12-17T12:05:49.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:49.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] boot 2023-12-17T12:05:49.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: osdmap e37: 6 total, 6 up, 6 in 2023-12-17T12:05:49.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi192 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi192 to 8635M 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] boot 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: osdmap e37: 6 total, 6 up, 6 in 2023-12-17T12:05:49.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi192 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi192 to 8635M 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: osd.5 [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] boot 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: osdmap e37: 6 total, 6 up, 6 in 2023-12-17T12:05:49.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-17T12:05:50.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:49 smithi141 ceph-mon[41225]: osdmap e38: 6 total, 6 up, 6 in 2023-12-17T12:05:50.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:49 smithi141 ceph-mon[41225]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:50.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:49 smithi141 ceph-mon[41225]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:50.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:50.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:50.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:50.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:49 smithi192 ceph-mon[41899]: osdmap e38: 6 total, 6 up, 6 in 2023-12-17T12:05:50.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:49 smithi192 ceph-mon[41899]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:50.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:49 smithi192 ceph-mon[41899]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:50.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:50.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:50.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:50.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:49 smithi027 ceph-mon[36871]: osdmap e38: 6 total, 6 up, 6 in 2023-12-17T12:05:50.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:49 smithi027 ceph-mon[36871]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:50.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:49 smithi027 ceph-mon[36871]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:05:50.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:05:50.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:05:50.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:51.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:50 smithi141 ceph-mon[41225]: osdmap e39: 6 total, 6 up, 6 in 2023-12-17T12:05:51.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:50 smithi192 ceph-mon[41899]: osdmap e39: 6 total, 6 up, 6 in 2023-12-17T12:05:51.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:50 smithi027 ceph-mon[36871]: osdmap e39: 6 total, 6 up, 6 in 2023-12-17T12:05:52.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:51 smithi141 ceph-mon[41225]: osdmap e40: 6 total, 6 up, 6 in 2023-12-17T12:05:52.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:51 smithi141 ceph-mon[41225]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:52.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:51 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/446654054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a2aa985-c312-47be-b27a-59be5ce1ab30"}]: dispatch 2023-12-17T12:05:52.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:51 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/446654054' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a2aa985-c312-47be-b27a-59be5ce1ab30"}]': finished 2023-12-17T12:05:52.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:51 smithi141 ceph-mon[41225]: osdmap e41: 7 total, 6 up, 7 in 2023-12-17T12:05:52.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:05:52.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:51 smithi192 ceph-mon[41899]: osdmap e40: 6 total, 6 up, 6 in 2023-12-17T12:05:52.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:51 smithi192 ceph-mon[41899]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:52.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:51 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/446654054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a2aa985-c312-47be-b27a-59be5ce1ab30"}]: dispatch 2023-12-17T12:05:52.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:51 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/446654054' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a2aa985-c312-47be-b27a-59be5ce1ab30"}]': finished 2023-12-17T12:05:52.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:51 smithi192 ceph-mon[41899]: osdmap e41: 7 total, 6 up, 7 in 2023-12-17T12:05:52.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:05:52.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:51 smithi027 ceph-mon[36871]: osdmap e40: 6 total, 6 up, 6 in 2023-12-17T12:05:52.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:51 smithi027 ceph-mon[36871]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:52.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:51 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/446654054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a2aa985-c312-47be-b27a-59be5ce1ab30"}]: dispatch 2023-12-17T12:05:52.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:51 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/446654054' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a2aa985-c312-47be-b27a-59be5ce1ab30"}]': finished 2023-12-17T12:05:52.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:51 smithi027 ceph-mon[36871]: osdmap e41: 7 total, 6 up, 7 in 2023-12-17T12:05:52.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:05:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:52 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3434060784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:53.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:52 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3434060784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:53.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:52 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3434060784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:05:54.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:53 smithi141 ceph-mon[41225]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:54.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:53 smithi192 ceph-mon[41899]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:54.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:53 smithi027 ceph-mon[36871]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:05:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:55 smithi027 ceph-mon[36871]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:05:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:55 smithi141 ceph-mon[41225]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:05:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:55 smithi192 ceph-mon[41899]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:05:58.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:57 smithi027 ceph-mon[36871]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-12-17T12:05:58.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:57 smithi141 ceph-mon[41225]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-12-17T12:05:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:57 smithi192 ceph-mon[41899]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-12-17T12:05:59.148 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:58 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-17T12:05:59.148 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:58 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:59.148 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:58 smithi027 ceph-mon[36871]: Deploying daemon osd.6 on smithi027 2023-12-17T12:05:59.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:58 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-17T12:05:59.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:58 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:59.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:58 smithi141 ceph-mon[41225]: Deploying daemon osd.6 on smithi027 2023-12-17T12:05:59.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:58 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-17T12:05:59.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:58 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:05:59.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:58 smithi192 ceph-mon[41899]: Deploying daemon osd.6 on smithi027 2023-12-17T12:06:00.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:05:59 smithi027 ceph-mon[36871]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-17T12:06:00.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:05:59 smithi141 ceph-mon[41225]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-17T12:06:00.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:05:59 smithi192 ceph-mon[41899]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-17T12:06:02.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:01 smithi027 ceph-mon[36871]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-17T12:06:02.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:01 smithi141 ceph-mon[41225]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-17T12:06:02.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:01 smithi192 ceph-mon[41899]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-17T12:06:04.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:04 smithi027 ceph-mon[36871]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-17T12:06:04.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:04 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:04.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:04 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:04.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:04 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:05.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:04 smithi141 ceph-mon[41225]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-17T12:06:05.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:04 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:05.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:04 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:05.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:04 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:05.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:04 smithi192 ceph-mon[41899]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-17T12:06:05.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:04 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:05.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:04 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:05.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:04 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.592+0000 7f36d37fe700 1 -- 172.21.15.27:0/3091593879 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f36dc074f20 con 0x7f36c80a1500 2023-12-17T12:06:06.594 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 6 on host 'smithi027' 2023-12-17T12:06:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.595+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36c80a1500 msgr2=0x7f36c80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.595+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36c80a1500 0x7f36c80a39c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36dc102680 tx=0x7f36d8009310 comp rx=0 tx=0).stop 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.595+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 msgr2=0x7f36dc103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.595+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 0x7f36dc103ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f36d4008c80 tx=0x7f36d4004f90 comp rx=0 tx=0).stop 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.596+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 shutdown_connections 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.596+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36c80a1500 0x7f36c80a39c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.596+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc101620 0x7f36dc107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.596+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36dc1031c0 0x7f36dc103ee0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.596+0000 7f36e44a3700 1 --2- 172.21.15.27:0/3091593879 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f36dc102820 0x7f36dc1078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.596+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 >> 172.21.15.27:0/3091593879 conn(0x7f36dc0fcbe0 msgr2=0x7f36dc073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.596+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 shutdown_connections 2023-12-17T12:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:06.596+0000 7f36e44a3700 1 -- 172.21.15.27:0/3091593879 wait complete. 2023-12-17T12:06:06.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:06:06.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:06.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:06.881 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:06.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:06.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:06.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:06.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:06.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:06 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:07.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:06:07.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:07.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:07.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:06 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:07.055 DEBUG:teuthology.orchestra.run.smithi027:osd.6> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.6.service 2023-12-17T12:06:07.060 INFO:tasks.cephadm:Deploying osd.7 on smithi141 with /dev/vg_nvme/lv_2... 2023-12-17T12:06:07.060 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:07.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:06 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:07.144 INFO:journalctl@ceph.osd.6.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:06:07.469 INFO:teuthology.orchestra.run.smithi141.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.b/config 2023-12-17T12:06:07.674 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:07 smithi027 ceph-mon[36871]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-17T12:06:07.674 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:06:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:06:07.421+0000 7f83d27ee640 -1 osd.6 0 log_to_monitors true 2023-12-17T12:06:07.966 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:07 smithi141 ceph-mon[41225]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-17T12:06:08.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:07 smithi192 ceph-mon[41899]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-17T12:06:08.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:08 smithi141 ceph-mon[41225]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:06:08.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:08 smithi141 ceph-mon[41225]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-17T12:06:08.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:08 smithi141 ceph-mon[41225]: osdmap e42: 7 total, 6 up, 7 in 2023-12-17T12:06:08.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:08 smithi141 ceph-mon[41225]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:06:08.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:08 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:08.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:08 smithi027 ceph-mon[36871]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:06:08.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:08 smithi027 ceph-mon[36871]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-17T12:06:08.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:08 smithi027 ceph-mon[36871]: osdmap e42: 7 total, 6 up, 7 in 2023-12-17T12:06:08.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:08 smithi027 ceph-mon[36871]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:06:08.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:08 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:08.913 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:06:08 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:06:08.659+0000 7f83c7e65700 -1 osd.6 0 waiting for initial osdmap 2023-12-17T12:06:08.913 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:06:08 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:06:08.670+0000 7f83bfc57700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:06:09.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:08 smithi192 ceph-mon[41899]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:06:09.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:08 smithi192 ceph-mon[41899]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-17T12:06:09.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:08 smithi192 ceph-mon[41899]: osdmap e42: 7 total, 6 up, 7 in 2023-12-17T12:06:09.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:08 smithi192 ceph-mon[41899]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:06:09.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:08 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:09.218 INFO:teuthology.orchestra.run.smithi141.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-17T12:06:09.219 INFO:teuthology.orchestra.run.smithi141.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-17T12:06:09.219 INFO:teuthology.orchestra.run.smithi141.stderr: stderr: 10+0 records in 2023-12-17T12:06:09.219 INFO:teuthology.orchestra.run.smithi141.stderr:10+0 records out 2023-12-17T12:06:09.219 INFO:teuthology.orchestra.run.smithi141.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0318969 s, 329 MB/s 2023-12-17T12:06:09.219 INFO:teuthology.orchestra.run.smithi141.stderr:--> Zapping successful for: 2023-12-17T12:06:09.608 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi141:vg_nvme/lv_2 2023-12-17T12:06:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:09 smithi027 ceph-mon[36871]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:06:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:09 smithi027 ceph-mon[36871]: osdmap e43: 7 total, 6 up, 7 in 2023-12-17T12:06:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:09 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:09 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:09 smithi141 ceph-mon[41225]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:06:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:09 smithi141 ceph-mon[41225]: osdmap e43: 7 total, 6 up, 7 in 2023-12-17T12:06:10.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:09 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:10.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:09 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:10.015 INFO:teuthology.orchestra.run.smithi141.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.b/config 2023-12-17T12:06:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:09 smithi192 ceph-mon[41899]: from='osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:06:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:09 smithi192 ceph-mon[41899]: osdmap e43: 7 total, 6 up, 7 in 2023-12-17T12:06:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:09 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:09 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:10.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:06:10.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:06:10.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] boot 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: osdmap e44: 7 total, 7 up, 7 in 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:10.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:06:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:06:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:06:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] boot 2023-12-17T12:06:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: osdmap e44: 7 total, 7 up, 7 in 2023-12-17T12:06:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.149 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3121983142 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28101620 msgr2=0x7fdc28101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:11.150 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3121983142 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28101620 0x7fdc28101a40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdc240099c0 tx=0x7fdc24009cd0 comp rx=0 tx=0).stop 2023-12-17T12:06:11.150 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3121983142 shutdown_connections 2023-12-17T12:06:11.150 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3121983142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc281031c0 0x7fdc28109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:11.150 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3121983142 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28101620 0x7fdc28101a40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:11.150 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3121983142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc28102820 0x7fdc28102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:11.150 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3121983142 >> 172.21.15.141:0/3121983142 conn(0x7fdc280fcbe0 msgr2=0x7fdc280ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:06:11.150 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3121983142 shutdown_connections 2023-12-17T12:06:11.150 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.147+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3121983142 wait complete. 2023-12-17T12:06:11.152 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.148+0000 7fdc2f5fa700 1 Processor -- start 2023-12-17T12:06:11.152 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.148+0000 7fdc2f5fa700 1 -- start start 2023-12-17T12:06:11.152 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.148+0000 7fdc2f5fa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc28101620 0x7fdc28075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:11.152 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.148+0000 7fdc2f5fa700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28102820 0x7fdc280739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:11.152 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.148+0000 7fdc2f5fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc281031c0 0x7fdc28073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:11.152 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.148+0000 7fdc2f5fa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280746a0 con 0x7fdc281031c0 2023-12-17T12:06:11.152 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.148+0000 7fdc2f5fa700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28074810 con 0x7fdc28102820 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2f5fa700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc281a1580 con 0x7fdc28101620 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28102820 0x7fdc280739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28102820 0x7fdc280739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:50374/0 (socket says 172.21.15.141:50374) 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 -- 172.21.15.141:0/3455590925 learned_addr learned my addr 172.21.15.141:0/3455590925 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2db97700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc281031c0 0x7fdc28073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2d396700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc28101620 0x7fdc28075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 -- 172.21.15.141:0/3455590925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc28101620 msgr2=0x7fdc28075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc28101620 0x7fdc28075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 -- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc281031c0 msgr2=0x7fdc28073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc281031c0 0x7fdc28073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 -- 172.21.15.141:0/3455590925 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240095f0 con 0x7fdc28102820 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2db97700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc281031c0 0x7fdc28073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2d396700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc28101620 0x7fdc28075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:06:11.153 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.149+0000 7fdc2cb95700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28102820 0x7fdc280739c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d950 tx=0x7fdc1800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:06:11.154 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.150+0000 7fdc1e7fc700 1 -- 172.21.15.141:0/3455590925 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18009b40 con 0x7fdc28102820 2023-12-17T12:06:11.154 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.150+0000 7fdc1e7fc700 1 -- 172.21.15.141:0/3455590925 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc18009ca0 con 0x7fdc28102820 2023-12-17T12:06:11.154 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.150+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc281a1810 con 0x7fdc28102820 2023-12-17T12:06:11.154 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.150+0000 7fdc1e7fc700 1 -- 172.21.15.141:0/3455590925 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800f620 con 0x7fdc28102820 2023-12-17T12:06:11.154 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.150+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281a1d60 con 0x7fdc28102820 2023-12-17T12:06:11.154 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.151+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc28108ba0 con 0x7fdc28102820 2023-12-17T12:06:11.155 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.153+0000 7fdc1e7fc700 1 -- 172.21.15.141:0/3455590925 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fdc18010460 con 0x7fdc28102820 2023-12-17T12:06:11.157 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.155+0000 7fdc1e7fc700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fdc140a1500 0x7fdc140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:11.157 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.155+0000 7fdc1e7fc700 1 -- 172.21.15.141:0/3455590925 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fdc180cc3b0 con 0x7fdc28102820 2023-12-17T12:06:11.157 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.155+0000 7fdc2d396700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fdc140a1500 0x7fdc140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:11.158 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.156+0000 7fdc2d396700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fdc140a1500 0x7fdc140a39c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000c00 tx=0x7fdc24005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.160+0000 7fdc1e7fc700 1 -- 172.21.15.141:0/3455590925 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdc18096680 con 0x7fdc28102820 2023-12-17T12:06:11.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:06:11.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:06:11.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-17T12:06:11.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: osd.6 [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] boot 2023-12-17T12:06:11.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: osdmap e44: 7 total, 7 up, 7 in 2023-12-17T12:06:11.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:11.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:11.367 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:11.365+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc28061a10 con 0x7fdc140a1500 2023-12-17T12:06:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:11 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi027 2023-12-17T12:06:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:11 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi027 to 5757M 2023-12-17T12:06:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:11 smithi141 ceph-mon[41225]: osdmap e45: 7 total, 7 up, 7 in 2023-12-17T12:06:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:11 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:11 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:11 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:11 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi027 2023-12-17T12:06:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:11 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi027 to 5757M 2023-12-17T12:06:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:11 smithi192 ceph-mon[41899]: osdmap e45: 7 total, 7 up, 7 in 2023-12-17T12:06:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:12.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:11 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi027 2023-12-17T12:06:12.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:11 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi027 to 5757M 2023-12-17T12:06:12.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:11 smithi027 ceph-mon[36871]: osdmap e45: 7 total, 7 up, 7 in 2023-12-17T12:06:12.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:12.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:12.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:12.671 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:12 smithi141 ceph-mon[41225]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:13.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:12 smithi141 ceph-mon[41225]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:12 smithi192 ceph-mon[41899]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:12 smithi192 ceph-mon[41899]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:13.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:12 smithi027 ceph-mon[36871]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:13.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:12 smithi027 ceph-mon[36871]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:15.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:14 smithi141 ceph-mon[41225]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:15.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:14 smithi141 ceph-mon[41225]: from='client.? 172.21.15.141:0/3287685926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]: dispatch 2023-12-17T12:06:15.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:14 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]: dispatch 2023-12-17T12:06:15.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:14 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]': finished 2023-12-17T12:06:15.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:14 smithi141 ceph-mon[41225]: osdmap e46: 8 total, 7 up, 8 in 2023-12-17T12:06:15.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:15.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:14 smithi141 ceph-mon[41225]: from='client.? 172.21.15.141:0/1322975683' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:06:15.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:14 smithi192 ceph-mon[41899]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:15.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:14 smithi192 ceph-mon[41899]: from='client.? 172.21.15.141:0/3287685926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]: dispatch 2023-12-17T12:06:15.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:14 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]: dispatch 2023-12-17T12:06:15.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:14 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]': finished 2023-12-17T12:06:15.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:14 smithi192 ceph-mon[41899]: osdmap e46: 8 total, 7 up, 8 in 2023-12-17T12:06:15.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:15.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:14 smithi192 ceph-mon[41899]: from='client.? 172.21.15.141:0/1322975683' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:06:15.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:14 smithi027 ceph-mon[36871]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:15.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:14 smithi027 ceph-mon[36871]: from='client.? 172.21.15.141:0/3287685926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]: dispatch 2023-12-17T12:06:15.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:14 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]: dispatch 2023-12-17T12:06:15.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:14 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdadd696-7e4a-411e-8366-17d9cb235d00"}]': finished 2023-12-17T12:06:15.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:14 smithi027 ceph-mon[36871]: osdmap e46: 8 total, 7 up, 8 in 2023-12-17T12:06:15.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:15.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:14 smithi027 ceph-mon[36871]: from='client.? 172.21.15.141:0/1322975683' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:06:17.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:16 smithi141 ceph-mon[41225]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:17.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:16 smithi192 ceph-mon[41899]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:17.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:16 smithi027 ceph-mon[36871]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:18.943 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:18 smithi141 ceph-mon[41225]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:18 smithi192 ceph-mon[41899]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:19.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:18 smithi027 ceph-mon[36871]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:20.693 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:20 smithi141 ceph-mon[41225]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:20.693 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:20 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-17T12:06:20.694 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:20 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:21.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:20 smithi192 ceph-mon[41899]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:21.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-17T12:06:21.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:20 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:21.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:20 smithi027 ceph-mon[36871]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:21.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-17T12:06:21.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:20 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:22.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:21 smithi141 ceph-mon[41225]: Deploying daemon osd.7 on smithi141 2023-12-17T12:06:22.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:21 smithi192 ceph-mon[41899]: Deploying daemon osd.7 on smithi141 2023-12-17T12:06:22.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:21 smithi027 ceph-mon[36871]: Deploying daemon osd.7 on smithi141 2023-12-17T12:06:22.889 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:22 smithi141 ceph-mon[41225]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:23.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:22 smithi192 ceph-mon[41899]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:23.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:22 smithi027 ceph-mon[36871]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:24.837 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:24 smithi141 ceph-mon[41225]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:25.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:24 smithi192 ceph-mon[41899]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:25.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:24 smithi027 ceph-mon[36871]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:26.762 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:26 smithi141 ceph-mon[41225]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:26.762 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:26.762 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:26.762 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:26.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:26 smithi192 ceph-mon[41899]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:26.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:26.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:26.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:26 smithi027 ceph-mon[36871]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.440 INFO:teuthology.orchestra.run.smithi141.stdout:Created osd(s) 7 on host 'smithi141' 2023-12-17T12:06:28.440 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.438+0000 7fdc1e7fc700 1 -- 172.21.15.141:0/3455590925 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdc28061a10 con 0x7fdc140a1500 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fdc140a1500 msgr2=0x7fdc140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fdc140a1500 0x7fdc140a39c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000c00 tx=0x7fdc24005fb0 comp rx=0 tx=0).stop 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28102820 msgr2=0x7fdc280739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28102820 0x7fdc280739c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d950 tx=0x7fdc1800dd10 comp rx=0 tx=0).stop 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 shutdown_connections 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fdc140a1500 0x7fdc140a39c0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000c00 tx=0x7fdc24005fb0 comp rx=0 tx=0).stop 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc28101620 0x7fdc28075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc281031c0 0x7fdc28073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 --2- 172.21.15.141:0/3455590925 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fdc28102820 0x7fdc280739c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:28.443 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.441+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 >> 172.21.15.141:0/3455590925 conn(0x7fdc280fcbe0 msgr2=0x7fdc28105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:06:28.444 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.442+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 shutdown_connections 2023-12-17T12:06:28.444 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:06:28.442+0000 7fdc2f5fa700 1 -- 172.21.15.141:0/3455590925 wait complete. 2023-12-17T12:06:28.645 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:28 smithi141 ceph-mon[41225]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:28.645 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.645 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.645 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:28.645 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:28.646 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:28 smithi027 ceph-mon[36871]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:28 smithi192 ceph-mon[41899]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:28.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:28.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:28.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:28.841 DEBUG:teuthology.orchestra.run.smithi141:osd.7> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.7.service 2023-12-17T12:06:28.846 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2023-12-17T12:06:28.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-17T12:06:28.929 INFO:journalctl@ceph.osd.7.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:06:29.257 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:06:29.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:29 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:29.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:29 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:29.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:29 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:29 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:29 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:29 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:29.752 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:29 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:29.752 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:29 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:29.752 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:29 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:29.795 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:06:29 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:06:29.528+0000 7f7ab4627640 -1 osd.7 0 log_to_monitors true 2023-12-17T12:06:30.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:30 smithi192 ceph-mon[41899]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:30.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:30 smithi192 ceph-mon[41899]: from='osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-17T12:06:30.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:30 smithi192 ceph-mon[41899]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-17T12:06:30.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:30 smithi027 ceph-mon[36871]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:30.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:30 smithi027 ceph-mon[36871]: from='osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-17T12:06:30.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:30 smithi027 ceph-mon[36871]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-17T12:06:30.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:30 smithi141 ceph-mon[41225]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:30.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:30 smithi141 ceph-mon[41225]: from='osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-17T12:06:30.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:30 smithi141 ceph-mon[41225]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-17T12:06:31.089 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-17T12:06:31.089 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-17T12:06:31.089 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-12-17T12:06:31.089 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-12-17T12:06:31.090 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0430895 s, 243 MB/s 2023-12-17T12:06:31.090 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-12-17T12:06:31.468 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:06:31.456+0000 7f7aa9c9e700 -1 osd.7 0 waiting for initial osdmap 2023-12-17T12:06:31.469 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-mon[41225]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-17T12:06:31.469 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-mon[41225]: from='osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:06:31.469 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-mon[41225]: osdmap e47: 8 total, 7 up, 8 in 2023-12-17T12:06:31.469 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:31.469 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-mon[41225]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:06:31.469 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-mon[41225]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:06:31.469 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-mon[41225]: osdmap e48: 8 total, 7 up, 8 in 2023-12-17T12:06:31.469 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:31.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2023-12-17T12:06:31.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:06:31 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:06:31.467+0000 7f7aa5a98700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:06:31.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:31 smithi192 ceph-mon[41899]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-17T12:06:31.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:31 smithi192 ceph-mon[41899]: from='osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:06:31.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:31 smithi192 ceph-mon[41899]: osdmap e47: 8 total, 7 up, 8 in 2023-12-17T12:06:31.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:31.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:31 smithi192 ceph-mon[41899]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:06:31.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:31 smithi192 ceph-mon[41899]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:06:31.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:31 smithi192 ceph-mon[41899]: osdmap e48: 8 total, 7 up, 8 in 2023-12-17T12:06:31.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:31 smithi027 ceph-mon[36871]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-17T12:06:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:31 smithi027 ceph-mon[36871]: from='osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:06:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:31 smithi027 ceph-mon[36871]: osdmap e47: 8 total, 7 up, 8 in 2023-12-17T12:06:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:31 smithi027 ceph-mon[36871]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:06:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:31 smithi027 ceph-mon[36871]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:06:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:31 smithi027 ceph-mon[36871]: osdmap e48: 8 total, 7 up, 8 in 2023-12-17T12:06:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:31.921 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:06:32.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:32.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:32.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi141 2023-12-17T12:06:32.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi141 to 5757M 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] boot 2023-12-17T12:06:32.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: osdmap e49: 8 total, 8 up, 8 in 2023-12-17T12:06:32.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:32 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi141 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi141 to 5757M 2023-12-17T12:06:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:32.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:32.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] boot 2023-12-17T12:06:32.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: osdmap e49: 8 total, 8 up, 8 in 2023-12-17T12:06:32.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:32 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi141 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi141 to 5757M 2023-12-17T12:06:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:32.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:32.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:32.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: osd.7 [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] boot 2023-12-17T12:06:32.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: osdmap e49: 8 total, 8 up, 8 in 2023-12-17T12:06:32.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:32 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.137+0000 7fc08b2c8700 1 -- 172.21.15.192:0/366291263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 msgr2=0x7fc084102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.137+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/366291263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 0x7fc084102c60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc074009990 tx=0x7fc074009ca0 comp rx=0 tx=0).stop 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.138+0000 7fc08b2c8700 1 -- 172.21.15.192:0/366291263 shutdown_connections 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.138+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/366291263 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc0841031a0 0x7fc084109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.138+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/366291263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc084101600 0x7fc084101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.138+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/366291263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 0x7fc084102c60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.138+0000 7fc08b2c8700 1 -- 172.21.15.192:0/366291263 >> 172.21.15.192:0/366291263 conn(0x7fc0840fcba0 msgr2=0x7fc0840fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.138+0000 7fc08b2c8700 1 -- 172.21.15.192:0/366291263 shutdown_connections 2023-12-17T12:06:33.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.138+0000 7fc08b2c8700 1 -- 172.21.15.192:0/366291263 wait complete. 2023-12-17T12:06:33.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc08b2c8700 1 Processor -- start 2023-12-17T12:06:33.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc08b2c8700 1 -- start start 2023-12-17T12:06:33.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc08b2c8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc084101600 0x7fc0840752d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:33.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc08b2c8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 0x7fc0840739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:33.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc08b2c8700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc0841031a0 0x7fc084073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:33.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc08b2c8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840745e0 con 0x7fc084101600 2023-12-17T12:06:33.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc08b2c8700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc084074750 con 0x7fc0841031a0 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc08b2c8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0841a1540 con 0x7fc084102800 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc088863700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 0x7fc0840739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc088863700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 0x7fc0840739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:60820/0 (socket says 172.21.15.192:60820) 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.139+0000 7fc088863700 1 -- 172.21.15.192:0/87478462 learned_addr learned my addr 172.21.15.192:0/87478462 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc089064700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc084101600 0x7fc0840752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc089865700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc0841031a0 0x7fc084073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc088863700 1 -- 172.21.15.192:0/87478462 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc0841031a0 msgr2=0x7fc084073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc088863700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc0841031a0 0x7fc084073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc088863700 1 -- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc084101600 msgr2=0x7fc0840752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc088863700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc084101600 0x7fc0840752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc088863700 1 -- 172.21.15.192:0/87478462 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0740095f0 con 0x7fc084102800 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc089865700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc0841031a0 0x7fc084073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:06:33.142 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc088863700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 0x7fc0840739c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc0740098e0 tx=0x7fc07400bb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:06:33.143 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc07a7fc700 1 -- 172.21.15.192:0/87478462 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07401d070 con 0x7fc084102800 2023-12-17T12:06:33.143 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc07a7fc700 1 -- 172.21.15.192:0/87478462 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc074003da0 con 0x7fc084102800 2023-12-17T12:06:33.143 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0841a1740 con 0x7fc084102800 2023-12-17T12:06:33.143 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.140+0000 7fc07a7fc700 1 -- 172.21.15.192:0/87478462 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074021410 con 0x7fc084102800 2023-12-17T12:06:33.143 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.141+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0841a1b50 con 0x7fc084102800 2023-12-17T12:06:33.143 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.142+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc084108b30 con 0x7fc084102800 2023-12-17T12:06:33.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.144+0000 7fc07a7fc700 1 -- 172.21.15.192:0/87478462 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc07400f460 con 0x7fc084102800 2023-12-17T12:06:33.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.146+0000 7fc07a7fc700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc0700a1480 0x7fc0700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:33.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.146+0000 7fc07a7fc700 1 -- 172.21.15.192:0/87478462 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5430+0+0 (secure 0 0 0) 0x7fc0740cdad0 con 0x7fc084102800 2023-12-17T12:06:33.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.146+0000 7fc089064700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc0700a1480 0x7fc0700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:33.148 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.147+0000 7fc089064700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc0700a1480 0x7fc0700a3940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc080005950 tx=0x7fc0800058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:06:33.150 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.149+0000 7fc07a7fc700 1 -- 172.21.15.192:0/87478462 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc074097c60 con 0x7fc084102800 2023-12-17T12:06:33.357 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:33.356+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc08402d380 con 0x7fc0700a1480 2023-12-17T12:06:33.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:33 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:06:33.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:33 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:06:33.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:33 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:33.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:33 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:33.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:33 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:33.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:33 smithi141 ceph-mon[41225]: osdmap e50: 8 total, 8 up, 8 in 2023-12-17T12:06:33.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:33 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:06:33.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:33 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:06:33.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:33 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:33.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:33 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:33.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:33 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:33.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:33 smithi192 ceph-mon[41899]: osdmap e50: 8 total, 8 up, 8 in 2023-12-17T12:06:33.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:33 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:06:33.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:33 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:06:33.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:33 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:33.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:33 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:33.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:33 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:33.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:33 smithi027 ceph-mon[36871]: osdmap e50: 8 total, 8 up, 8 in 2023-12-17T12:06:34.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:34 smithi141 ceph-mon[41225]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:34.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:34 smithi141 ceph-mon[41225]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:34.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:34 smithi141 ceph-mon[41225]: osdmap e51: 8 total, 8 up, 8 in 2023-12-17T12:06:34.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:34 smithi192 ceph-mon[41899]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:34.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:34 smithi192 ceph-mon[41899]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:34.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:34 smithi192 ceph-mon[41899]: osdmap e51: 8 total, 8 up, 8 in 2023-12-17T12:06:34.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:34 smithi027 ceph-mon[36871]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:34.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:34 smithi027 ceph-mon[36871]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:34.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:34 smithi027 ceph-mon[36871]: osdmap e51: 8 total, 8 up, 8 in 2023-12-17T12:06:36.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:36 smithi141 ceph-mon[41225]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:36.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:36 smithi141 ceph-mon[41225]: osdmap e52: 8 total, 8 up, 8 in 2023-12-17T12:06:36.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:36 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/1742480301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]: dispatch 2023-12-17T12:06:36.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:36 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]: dispatch 2023-12-17T12:06:36.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:36 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]': finished 2023-12-17T12:06:36.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:36 smithi141 ceph-mon[41225]: osdmap e53: 9 total, 8 up, 9 in 2023-12-17T12:06:36.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:36.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:36 smithi192 ceph-mon[41899]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:36.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:36 smithi192 ceph-mon[41899]: osdmap e52: 8 total, 8 up, 8 in 2023-12-17T12:06:36.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:36 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/1742480301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]: dispatch 2023-12-17T12:06:36.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:36 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]: dispatch 2023-12-17T12:06:36.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:36 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]': finished 2023-12-17T12:06:36.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:36 smithi192 ceph-mon[41899]: osdmap e53: 9 total, 8 up, 9 in 2023-12-17T12:06:36.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:36.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:36 smithi027 ceph-mon[36871]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:36.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:36 smithi027 ceph-mon[36871]: osdmap e52: 8 total, 8 up, 8 in 2023-12-17T12:06:36.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:36 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/1742480301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]: dispatch 2023-12-17T12:06:36.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:36 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]: dispatch 2023-12-17T12:06:36.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:36 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92586b7e-b4c6-4e69-aecb-e7696a821a2f"}]': finished 2023-12-17T12:06:36.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:36 smithi027 ceph-mon[36871]: osdmap e53: 9 total, 8 up, 9 in 2023-12-17T12:06:36.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:37.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:37 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/3372350520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:06:37.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:37 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/3372350520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:06:37.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:37 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/3372350520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:06:38.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:38 smithi141 ceph-mon[41225]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:38.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:38 smithi192 ceph-mon[41899]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:38.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:38 smithi027 ceph-mon[36871]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:40.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:40 smithi192 ceph-mon[41899]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:40.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:40 smithi141 ceph-mon[41225]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:40.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:40 smithi027 ceph-mon[36871]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:42.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:42 smithi192 ceph-mon[41899]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:42.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:42 smithi141 ceph-mon[41225]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:42.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:42 smithi027 ceph-mon[36871]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:43.559 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:43 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-17T12:06:43.559 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:43 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:43.559 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:43 smithi192 ceph-mon[41899]: Deploying daemon osd.8 on smithi192 2023-12-17T12:06:43.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:43 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-17T12:06:43.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:43 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:43.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:43 smithi141 ceph-mon[41225]: Deploying daemon osd.8 on smithi192 2023-12-17T12:06:43.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-17T12:06:43.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:43.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:43 smithi027 ceph-mon[36871]: Deploying daemon osd.8 on smithi192 2023-12-17T12:06:44.718 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:44 smithi192 ceph-mon[41899]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:44.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:44 smithi141 ceph-mon[41225]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:44.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:44 smithi027 ceph-mon[36871]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:46.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:46 smithi141 ceph-mon[41225]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:46.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:46 smithi192 ceph-mon[41899]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:46.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:46 smithi027 ceph-mon[36871]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:48.545 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:48 smithi192 ceph-mon[41899]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:48.545 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:48.545 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:48.545 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:48 smithi141 ceph-mon[41225]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:48.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:48 smithi027 ceph-mon[36871]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:48.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:48.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:48.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:50.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:50 smithi192 ceph-mon[41899]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:50.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:50.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:50.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:50.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:50.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:50 smithi141 ceph-mon[41225]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:50.782 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.781+0000 7fc07a7fc700 1 -- 172.21.15.192:0/87478462 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc08402d380 con 0x7fc0700a1480 2023-12-17T12:06:50.782 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2023-12-17T12:06:50.784 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc0700a1480 msgr2=0x7fc0700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:50.784 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc0700a1480 0x7fc0700a3940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc080005950 tx=0x7fc0800058e0 comp rx=0 tx=0).stop 2023-12-17T12:06:50.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 msgr2=0x7fc0840739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:50.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 0x7fc0840739c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc0740098e0 tx=0x7fc07400bb00 comp rx=0 tx=0).stop 2023-12-17T12:06:50.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 shutdown_connections 2023-12-17T12:06:50.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc0700a1480 0x7fc0700a3940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:50.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc0841031a0 0x7fc084073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:50.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc084101600 0x7fc0840752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:50.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.783+0000 7fc08b2c8700 1 --2- 172.21.15.192:0/87478462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc084102800 0x7fc0840739c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:50.785 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.784+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 >> 172.21.15.192:0/87478462 conn(0x7fc0840fcba0 msgr2=0x7fc084105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:06:50.786 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.784+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 shutdown_connections 2023-12-17T12:06:50.786 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:06:50.784+0000 7fc08b2c8700 1 -- 172.21.15.192:0/87478462 wait complete. 2023-12-17T12:06:50.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:50 smithi027 ceph-mon[36871]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:50.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:50.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:50.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:50.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:50.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:51.224 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.8.service 2023-12-17T12:06:51.228 INFO:tasks.cephadm:Deploying osd.9 on smithi027 with /dev/vg_nvme/lv_1... 2023-12-17T12:06:51.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-17T12:06:51.307 INFO:journalctl@ceph.osd.8.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:06:51.662 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:06:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:51 smithi192 ceph-mon[41899]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:51 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:52.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:52.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:52.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:51 smithi141 ceph-mon[41225]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:52.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:51 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:52.150 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:52.150 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:52.150 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:51 smithi027 ceph-mon[36871]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:52.150 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:51 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:06:52.312 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:06:51 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:06:51.868+0000 7f1c1bab4640 -1 osd.8 0 log_to_monitors true 2023-12-17T12:06:52.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:52 smithi027 ceph-mon[36871]: from='osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-17T12:06:52.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:52 smithi027 ceph-mon[36871]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-17T12:06:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:52 smithi141 ceph-mon[41225]: from='osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-17T12:06:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:52 smithi141 ceph-mon[41225]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-17T12:06:53.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:52 smithi192 ceph-mon[41899]: from='osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-17T12:06:53.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:52 smithi192 ceph-mon[41899]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-17T12:06:53.446 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-17T12:06:53.446 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-17T12:06:53.446 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2023-12-17T12:06:53.446 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2023-12-17T12:06:53.446 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0311084 s, 337 MB/s 2023-12-17T12:06:53.446 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2023-12-17T12:06:53.869 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2023-12-17T12:06:54.043 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:06:53 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:06:53.792+0000 7f1c1112b700 -1 osd.8 0 waiting for initial osdmap 2023-12-17T12:06:54.043 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:06:53 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:06:53.804+0000 7f1c0a720700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:06:54.043 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:53 smithi192 ceph-mon[41899]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-17T12:06:54.043 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:53 smithi192 ceph-mon[41899]: from='osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:06:54.044 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:53 smithi192 ceph-mon[41899]: osdmap e54: 9 total, 8 up, 9 in 2023-12-17T12:06:54.044 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:53 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:54.044 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:53 smithi192 ceph-mon[41899]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:06:54.044 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:53 smithi192 ceph-mon[41899]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:54.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:53 smithi027 ceph-mon[36871]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-17T12:06:54.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:53 smithi027 ceph-mon[36871]: from='osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:06:54.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:53 smithi027 ceph-mon[36871]: osdmap e54: 9 total, 8 up, 9 in 2023-12-17T12:06:54.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:53 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:54.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:53 smithi027 ceph-mon[36871]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:06:54.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:53 smithi027 ceph-mon[36871]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:54.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:53 smithi141 ceph-mon[41225]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-17T12:06:54.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:53 smithi141 ceph-mon[41225]: from='osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:06:54.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:53 smithi141 ceph-mon[41225]: osdmap e54: 9 total, 8 up, 9 in 2023-12-17T12:06:54.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:53 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:54.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:53 smithi141 ceph-mon[41225]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:06:54.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:53 smithi141 ceph-mon[41225]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:54.307 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:06:55.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:06:55.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: osdmap e55: 9 total, 8 up, 9 in 2023-12-17T12:06:55.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] boot 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: osdmap e56: 9 total, 9 up, 9 in 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi192 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi192 to 5757M 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:55.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:54 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: osdmap e55: 9 total, 8 up, 9 in 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] boot 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: osdmap e56: 9 total, 9 up, 9 in 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi192 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi192 to 5757M 2023-12-17T12:06:55.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:55.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:55.164 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:54 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:06:55.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: osdmap e55: 9 total, 8 up, 9 in 2023-12-17T12:06:55.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: osd.8 [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] boot 2023-12-17T12:06:55.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: osdmap e56: 9 total, 9 up, 9 in 2023-12-17T12:06:55.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-17T12:06:55.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi192 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi192 to 5757M 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:06:55.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:54 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:06:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.490+0000 7f126f15d700 1 -- 172.21.15.27:0/913571950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12681031c0 msgr2=0x7f1268109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.490+0000 7f126f15d700 1 --2- 172.21.15.27:0/913571950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12681031c0 0x7f1268109a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f126000b0d0 tx=0x7f126000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:06:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.490+0000 7f126f15d700 1 -- 172.21.15.27:0/913571950 shutdown_connections 2023-12-17T12:06:55.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.490+0000 7f126f15d700 1 --2- 172.21.15.27:0/913571950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1268101620 0x7f1268101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:55.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.490+0000 7f126f15d700 1 --2- 172.21.15.27:0/913571950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12681031c0 0x7f1268109a70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:55.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.490+0000 7f126f15d700 1 --2- 172.21.15.27:0/913571950 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1268102820 0x7f1268102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:55.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.490+0000 7f126f15d700 1 -- 172.21.15.27:0/913571950 >> 172.21.15.27:0/913571950 conn(0x7f12680fcbe0 msgr2=0x7f12680ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:06:55.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.491+0000 7f126f15d700 1 -- 172.21.15.27:0/913571950 shutdown_connections 2023-12-17T12:06:55.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.491+0000 7f126f15d700 1 -- 172.21.15.27:0/913571950 wait complete. 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f126f15d700 1 Processor -- start 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f126f15d700 1 -- start start 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f126f15d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1268101620 0x7f1268199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f126f15d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1268102820 0x7f1268199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f126f15d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12681031c0 0x7f12681a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f126f15d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12681a1250 con 0x7f1268102820 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f126f15d700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12681a1390 con 0x7f12681031c0 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f126f15d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12681a1520 con 0x7f1268101620 2023-12-17T12:06:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.492+0000 7f1267fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1268102820 0x7f1268199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f126cef9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1268101620 0x7f1268199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1267fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1268102820 0x7f1268199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36922/0 (socket says 172.21.15.27:36922) 2023-12-17T12:06:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f126d6fa700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12681031c0 0x7f12681a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1267fff700 1 -- 172.21.15.27:0/154708089 learned_addr learned my addr 172.21.15.27:0/154708089 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1267fff700 1 -- 172.21.15.27:0/154708089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1268101620 msgr2=0x7f1268199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1267fff700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1268101620 0x7f1268199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1267fff700 1 -- 172.21.15.27:0/154708089 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12681031c0 msgr2=0x7f12681a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1267fff700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12681031c0 0x7f12681a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1267fff700 1 -- 172.21.15.27:0/154708089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1260009d00 con 0x7f1268102820 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f126d6fa700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12681031c0 0x7f12681a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1267fff700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1268102820 0x7f1268199c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f125400e9d0 tx=0x7f125400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.493+0000 7f1265ffb700 1 -- 172.21.15.27:0/154708089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254009790 con 0x7f1268102820 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.494+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12681a1800 con 0x7f1268102820 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.494+0000 7f1265ffb700 1 -- 172.21.15.27:0/154708089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f125400ee70 con 0x7f1268102820 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.494+0000 7f1265ffb700 1 -- 172.21.15.27:0/154708089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254018420 con 0x7f1268102820 2023-12-17T12:06:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.494+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12681a1d00 con 0x7f1268102820 2023-12-17T12:06:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.495+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126804f290 con 0x7f1268102820 2023-12-17T12:06:55.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.497+0000 7f1265ffb700 1 -- 172.21.15.27:0/154708089 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f125400cbd0 con 0x7f1268102820 2023-12-17T12:06:55.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.498+0000 7f1265ffb700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f12580a1480 0x7f12580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:06:55.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.498+0000 7f1265ffb700 1 -- 172.21.15.27:0/154708089 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f1254014070 con 0x7f1268102820 2023-12-17T12:06:55.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.499+0000 7f126cef9700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f12580a1480 0x7f12580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:06:55.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.499+0000 7f126cef9700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f12580a1480 0x7f12580a3940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1268102680 tx=0x7f125c007400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:06:55.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.502+0000 7f1265ffb700 1 -- 172.21.15.27:0/154708089 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1254096480 con 0x7f1268102820 2023-12-17T12:06:55.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:06:55.722+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f12681096e0 con 0x7f12580a1480 2023-12-17T12:06:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:55 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:06:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:55 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:06:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:55 smithi027 ceph-mon[36871]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:55 smithi027 ceph-mon[36871]: osdmap e57: 9 total, 9 up, 9 in 2023-12-17T12:06:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:56.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:55 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:06:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:55 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:06:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:55 smithi141 ceph-mon[41225]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:55 smithi141 ceph-mon[41225]: osdmap e57: 9 total, 9 up, 9 in 2023-12-17T12:06:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:55 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:06:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:55 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:06:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:55 smithi192 ceph-mon[41899]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-17T12:06:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:55 smithi192 ceph-mon[41899]: osdmap e57: 9 total, 9 up, 9 in 2023-12-17T12:06:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:06:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:06:56.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:06:57.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:56 smithi027 ceph-mon[36871]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:57.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:56 smithi027 ceph-mon[36871]: osdmap e58: 9 total, 9 up, 9 in 2023-12-17T12:06:57.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:56 smithi141 ceph-mon[41225]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:57.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:56 smithi141 ceph-mon[41225]: osdmap e58: 9 total, 9 up, 9 in 2023-12-17T12:06:57.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:56 smithi192 ceph-mon[41899]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:06:57.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:56 smithi192 ceph-mon[41899]: osdmap e58: 9 total, 9 up, 9 in 2023-12-17T12:06:58.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:57 smithi027 ceph-mon[36871]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:06:58.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:57 smithi027 ceph-mon[36871]: osdmap e59: 9 total, 9 up, 9 in 2023-12-17T12:06:58.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:57 smithi141 ceph-mon[41225]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:06:58.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:57 smithi141 ceph-mon[41225]: osdmap e59: 9 total, 9 up, 9 in 2023-12-17T12:06:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:57 smithi192 ceph-mon[41899]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:06:58.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:57 smithi192 ceph-mon[41899]: osdmap e59: 9 total, 9 up, 9 in 2023-12-17T12:06:59.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:58 smithi027 ceph-mon[36871]: osdmap e60: 9 total, 9 up, 9 in 2023-12-17T12:06:59.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:58 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2295682897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c11178c6-bb54-4043-a57c-e5c22a1560ef"}]: dispatch 2023-12-17T12:06:59.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:58 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2295682897' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c11178c6-bb54-4043-a57c-e5c22a1560ef"}]': finished 2023-12-17T12:06:59.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:58 smithi027 ceph-mon[36871]: osdmap e61: 10 total, 9 up, 10 in 2023-12-17T12:06:59.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:58 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:06:59.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:58 smithi141 ceph-mon[41225]: osdmap e60: 9 total, 9 up, 9 in 2023-12-17T12:06:59.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:58 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2295682897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c11178c6-bb54-4043-a57c-e5c22a1560ef"}]: dispatch 2023-12-17T12:06:59.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:58 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2295682897' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c11178c6-bb54-4043-a57c-e5c22a1560ef"}]': finished 2023-12-17T12:06:59.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:58 smithi141 ceph-mon[41225]: osdmap e61: 10 total, 9 up, 10 in 2023-12-17T12:06:59.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:58 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:06:59.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:58 smithi192 ceph-mon[41899]: osdmap e60: 9 total, 9 up, 9 in 2023-12-17T12:06:59.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:58 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2295682897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c11178c6-bb54-4043-a57c-e5c22a1560ef"}]: dispatch 2023-12-17T12:06:59.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:58 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2295682897' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c11178c6-bb54-4043-a57c-e5c22a1560ef"}]': finished 2023-12-17T12:06:59.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:58 smithi192 ceph-mon[41899]: osdmap e61: 10 total, 9 up, 10 in 2023-12-17T12:06:59.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:58 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:00.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:59 smithi027 ceph-mon[36871]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2023-12-17T12:07:00.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:06:59 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2558112661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:00.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:59 smithi141 ceph-mon[41225]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2023-12-17T12:07:00.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:06:59 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2558112661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:00.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:59 smithi192 ceph-mon[41899]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2023-12-17T12:07:00.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:06:59 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2558112661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:02.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:01 smithi027 ceph-mon[36871]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-12-17T12:07:02.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:01 smithi141 ceph-mon[41225]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-12-17T12:07:02.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:01 smithi192 ceph-mon[41899]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-12-17T12:07:04.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:03 smithi027 ceph-mon[36871]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:07:04.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:03 smithi141 ceph-mon[41225]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:07:04.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:03 smithi192 ceph-mon[41899]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-17T12:07:06.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:05 smithi027 ceph-mon[36871]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-12-17T12:07:06.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:05 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-17T12:07:06.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:05 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:06.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:05 smithi027 ceph-mon[36871]: Deploying daemon osd.9 on smithi027 2023-12-17T12:07:06.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:05 smithi141 ceph-mon[41225]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-12-17T12:07:06.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:05 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-17T12:07:06.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:05 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:06.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:05 smithi141 ceph-mon[41225]: Deploying daemon osd.9 on smithi027 2023-12-17T12:07:06.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:05 smithi192 ceph-mon[41899]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-12-17T12:07:06.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:05 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-17T12:07:06.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:05 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:06.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:05 smithi192 ceph-mon[41899]: Deploying daemon osd.9 on smithi027 2023-12-17T12:07:07.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:07 smithi027 ceph-mon[36871]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:08.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:07 smithi141 ceph-mon[41225]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:08.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:07 smithi192 ceph-mon[41899]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:10.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:09 smithi027 ceph-mon[36871]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:10.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:09 smithi141 ceph-mon[41225]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:10.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:09 smithi192 ceph-mon[41899]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:11.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:11.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:11.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:11 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:11 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:11 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:11 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:12.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:11 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:13.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:12 smithi141 ceph-mon[41225]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:12 smithi192 ceph-mon[41899]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:13.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:12 smithi027 ceph-mon[36871]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:13.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.984+0000 7f1265ffb700 1 -- 172.21.15.27:0/154708089 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f12681096e0 con 0x7f12580a1480 2023-12-17T12:07:13.985 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 9 on host 'smithi027' 2023-12-17T12:07:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.988+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f12580a1480 msgr2=0x7f12580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.988+0000 7f126f15d700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f12580a1480 0x7f12580a3940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1268102680 tx=0x7f125c007400 comp rx=0 tx=0).stop 2023-12-17T12:07:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.988+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1268102820 msgr2=0x7f1268199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.988+0000 7f126f15d700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1268102820 0x7f1268199c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f125400e9d0 tx=0x7f125400ece0 comp rx=0 tx=0).stop 2023-12-17T12:07:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.989+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 shutdown_connections 2023-12-17T12:07:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.989+0000 7f126f15d700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f12580a1480 0x7f12580a3940 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.989+0000 7f126f15d700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1268101620 0x7f1268199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.989+0000 7f126f15d700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12681031c0 0x7f12681a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.989+0000 7f126f15d700 1 --2- 172.21.15.27:0/154708089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1268102820 0x7f1268199c50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.989+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 >> 172.21.15.27:0/154708089 conn(0x7f12680fcbe0 msgr2=0x7f1268107fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:07:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.989+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 shutdown_connections 2023-12-17T12:07:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:07:13.989+0000 7f126f15d700 1 -- 172.21.15.27:0/154708089 wait complete. 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:14.354 DEBUG:teuthology.orchestra.run.smithi027:osd.9> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.9.service 2023-12-17T12:07:14.359 INFO:tasks.cephadm:Deploying osd.10 on smithi141 with /dev/vg_nvme/lv_1... 2023-12-17T12:07:14.359 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-17T12:07:14.427 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:14.427 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.427 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.427 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:14.427 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:14.427 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.428 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.428 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.428 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:14 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:14.437 INFO:journalctl@ceph.osd.9.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:14.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:14 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:14.773 INFO:teuthology.orchestra.run.smithi141.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.b/config 2023-12-17T12:07:15.055 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:07:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:07:14.793+0000 7f08ffb25640 -1 osd.9 0 log_to_monitors true 2023-12-17T12:07:15.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:15 smithi141 ceph-mon[41225]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-17T12:07:15.398 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:15 smithi027 ceph-mon[36871]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-17T12:07:15.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:15 smithi192 ceph-mon[41899]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-17T12:07:16.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:16 smithi027 ceph-mon[36871]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:16.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:16 smithi027 ceph-mon[36871]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-17T12:07:16.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:16 smithi027 ceph-mon[36871]: osdmap e62: 10 total, 9 up, 10 in 2023-12-17T12:07:16.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:16 smithi027 ceph-mon[36871]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:07:16.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:16 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:16.413 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:07:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:07:16.163+0000 7f08f519c700 -1 osd.9 0 waiting for initial osdmap 2023-12-17T12:07:16.413 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:07:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:07:16.184+0000 7f08f0795700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:07:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:16 smithi141 ceph-mon[41225]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:16 smithi141 ceph-mon[41225]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-17T12:07:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:16 smithi141 ceph-mon[41225]: osdmap e62: 10 total, 9 up, 10 in 2023-12-17T12:07:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:16 smithi141 ceph-mon[41225]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:07:16.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:16 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:16.527 INFO:teuthology.orchestra.run.smithi141.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-17T12:07:16.527 INFO:teuthology.orchestra.run.smithi141.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-17T12:07:16.527 INFO:teuthology.orchestra.run.smithi141.stderr: stderr: 10+0 records in 2023-12-17T12:07:16.527 INFO:teuthology.orchestra.run.smithi141.stderr:10+0 records out 2023-12-17T12:07:16.527 INFO:teuthology.orchestra.run.smithi141.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0392772 s, 267 MB/s 2023-12-17T12:07:16.527 INFO:teuthology.orchestra.run.smithi141.stderr:--> Zapping successful for: 2023-12-17T12:07:16.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:16 smithi192 ceph-mon[41899]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:16.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:16 smithi192 ceph-mon[41899]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-17T12:07:16.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:16 smithi192 ceph-mon[41899]: osdmap e62: 10 total, 9 up, 10 in 2023-12-17T12:07:16.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:16 smithi192 ceph-mon[41899]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2023-12-17T12:07:16.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:16 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:16.935 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi141:vg_nvme/lv_1 2023-12-17T12:07:17.344 INFO:teuthology.orchestra.run.smithi141.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.b/config 2023-12-17T12:07:17.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:17 smithi027 ceph-mon[36871]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:07:17.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:17 smithi027 ceph-mon[36871]: osdmap e63: 10 total, 9 up, 10 in 2023-12-17T12:07:17.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:17 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:17.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:17 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:17.503 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:17 smithi141 ceph-mon[41225]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:07:17.504 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:17 smithi141 ceph-mon[41225]: osdmap e63: 10 total, 9 up, 10 in 2023-12-17T12:07:17.504 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:17 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:17.504 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:17 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:17.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:17 smithi192 ceph-mon[41899]: from='osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2023-12-17T12:07:17.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:17 smithi192 ceph-mon[41899]: osdmap e63: 10 total, 9 up, 10 in 2023-12-17T12:07:17.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:17 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:17.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:17 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:18.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] boot 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: osdmap e64: 10 total, 10 up, 10 in 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi027 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi027 to 4317M 2023-12-17T12:07:18.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:18.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:18.516 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.547 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 -- 172.21.15.141:0/1304000085 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970102820 msgr2=0x7f1970102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:18.547 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 --2- 172.21.15.141:0/1304000085 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970102820 0x7f1970102c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f195c0099c0 tx=0x7f195c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:07:18.547 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 -- 172.21.15.141:0/1304000085 shutdown_connections 2023-12-17T12:07:18.548 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 --2- 172.21.15.141:0/1304000085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19701031c0 0x7f1970109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:18.548 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 --2- 172.21.15.141:0/1304000085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1970101620 0x7f1970101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:18.548 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 --2- 172.21.15.141:0/1304000085 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970102820 0x7f1970102c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:18.548 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 -- 172.21.15.141:0/1304000085 >> 172.21.15.141:0/1304000085 conn(0x7f19700fcbe0 msgr2=0x7f19700ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:07:18.548 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 -- 172.21.15.141:0/1304000085 shutdown_connections 2023-12-17T12:07:18.548 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.546+0000 7f1976ac6700 1 -- 172.21.15.141:0/1304000085 wait complete. 2023-12-17T12:07:18.551 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.547+0000 7f1976ac6700 1 Processor -- start 2023-12-17T12:07:18.551 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.547+0000 7f1976ac6700 1 -- start start 2023-12-17T12:07:18.551 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.547+0000 7f1976ac6700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970101620 0x7f19701996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:07:18.551 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.547+0000 7f1976ac6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1970102820 0x7f1970199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:07:18.551 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.547+0000 7f1976ac6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19701031c0 0x7f19701a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:07:18.551 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.547+0000 7f1976ac6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19701a1200 con 0x7f19701031c0 2023-12-17T12:07:18.551 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.547+0000 7f1976ac6700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19701a1340 con 0x7f1970101620 2023-12-17T12:07:18.551 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.547+0000 7f1976ac6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19701a14d0 con 0x7f1970102820 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970101620 0x7f19701996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970101620 0x7f19701996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:36790/0 (socket says 172.21.15.141:36790) 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 -- 172.21.15.141:0/3032123483 learned_addr learned my addr 172.21.15.141:0/3032123483 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1975063700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19701031c0 0x7f19701a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f196ffff700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1970102820 0x7f1970199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 -- 172.21.15.141:0/3032123483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1970102820 msgr2=0x7f1970199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1970102820 0x7f1970199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 -- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19701031c0 msgr2=0x7f19701a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19701031c0 0x7f19701a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 -- 172.21.15.141:0/3032123483 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0095f0 con 0x7f1970101620 2023-12-17T12:07:18.552 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f1974862700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970101620 0x7f19701996c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f196400d980 tx=0x7f196400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:07:18.553 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.548+0000 7f196dffb700 1 -- 172.21.15.141:0/3032123483 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400b950 con 0x7f1970101620 2023-12-17T12:07:18.553 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.549+0000 7f196dffb700 1 -- 172.21.15.141:0/3032123483 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1964007620 con 0x7f1970101620 2023-12-17T12:07:18.553 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.549+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19701a17b0 con 0x7f1970101620 2023-12-17T12:07:18.553 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.549+0000 7f196dffb700 1 -- 172.21.15.141:0/3032123483 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400f3c0 con 0x7f1970101620 2023-12-17T12:07:18.553 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.549+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19701a1d00 con 0x7f1970101620 2023-12-17T12:07:18.553 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.550+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197004f290 con 0x7f1970101620 2023-12-17T12:07:18.553 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.552+0000 7f196dffb700 1 -- 172.21.15.141:0/3032123483 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f196400bab0 con 0x7f1970101620 2023-12-17T12:07:18.554 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.553+0000 7f196dffb700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19600a1500 0x7f19600a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:07:18.554 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.553+0000 7f196dffb700 1 -- 172.21.15.141:0/3032123483 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f19640cc6f0 con 0x7f1970101620 2023-12-17T12:07:18.554 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.553+0000 7f196ffff700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19600a1500 0x7f19600a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:07:18.555 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.554+0000 7f196ffff700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19600a1500 0x7f19600a39c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c010 tx=0x7f195c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:07:18.558 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.557+0000 7f196dffb700 1 -- 172.21.15.141:0/3032123483 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f19640964b0 con 0x7f1970101620 2023-12-17T12:07:18.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:07:18.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] boot 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: osdmap e64: 10 total, 10 up, 10 in 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi027 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi027 to 4317M 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:18.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:18.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:07:18.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:07:18.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-17T12:07:18.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: osd.9 [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] boot 2023-12-17T12:07:18.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: osdmap e64: 10 total, 10 up, 10 in 2023-12-17T12:07:18.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-17T12:07:18.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi027 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi027 to 4317M 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:18.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:18.769 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:18.768+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1970109730 con 0x7f19600a1500 2023-12-17T12:07:19.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:19 smithi141 ceph-mon[41225]: osdmap e65: 10 total, 10 up, 10 in 2023-12-17T12:07:19.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:07:19.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:07:19.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:19 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:19.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:19 smithi027 ceph-mon[36871]: osdmap e65: 10 total, 10 up, 10 in 2023-12-17T12:07:19.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:07:19.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:07:19.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:19 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:20.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:19 smithi192 ceph-mon[41899]: osdmap e65: 10 total, 10 up, 10 in 2023-12-17T12:07:20.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:07:20.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:07:20.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:19 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:20.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:20 smithi027 ceph-mon[36871]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:07:20.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:20 smithi027 ceph-mon[36871]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:21.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:20 smithi141 ceph-mon[41225]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:07:21.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:20 smithi141 ceph-mon[41225]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:21.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:20 smithi192 ceph-mon[41899]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi141:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:07:21.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:20 smithi192 ceph-mon[41899]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:21.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:21 smithi027 ceph-mon[36871]: from='client.? 172.21.15.141:0/1714506399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]: dispatch 2023-12-17T12:07:21.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:21 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]: dispatch 2023-12-17T12:07:21.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:21 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]': finished 2023-12-17T12:07:21.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:21 smithi027 ceph-mon[36871]: osdmap e66: 11 total, 10 up, 11 in 2023-12-17T12:07:21.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:22.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:21 smithi141 ceph-mon[41225]: from='client.? 172.21.15.141:0/1714506399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]: dispatch 2023-12-17T12:07:22.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:21 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]: dispatch 2023-12-17T12:07:22.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:21 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]': finished 2023-12-17T12:07:22.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:21 smithi141 ceph-mon[41225]: osdmap e66: 11 total, 10 up, 11 in 2023-12-17T12:07:22.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:21 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:22.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:21 smithi192 ceph-mon[41899]: from='client.? 172.21.15.141:0/1714506399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]: dispatch 2023-12-17T12:07:22.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:21 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]: dispatch 2023-12-17T12:07:22.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:21 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2916e700-28e1-4b61-aedb-55dbd8b44208"}]': finished 2023-12-17T12:07:22.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:21 smithi192 ceph-mon[41899]: osdmap e66: 11 total, 10 up, 11 in 2023-12-17T12:07:22.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:22.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:22 smithi027 ceph-mon[36871]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:22.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:22 smithi027 ceph-mon[36871]: from='client.? 172.21.15.141:0/534227381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:23.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:22 smithi141 ceph-mon[41225]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:23.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:22 smithi141 ceph-mon[41225]: from='client.? 172.21.15.141:0/534227381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:23.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:22 smithi192 ceph-mon[41899]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:23.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:22 smithi192 ceph-mon[41899]: from='client.? 172.21.15.141:0/534227381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:24.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:24 smithi027 ceph-mon[36871]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:24 smithi141 ceph-mon[41225]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:25.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:24 smithi192 ceph-mon[41899]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:26 smithi027 ceph-mon[36871]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:27.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:26 smithi141 ceph-mon[41225]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:27.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:26 smithi192 ceph-mon[41899]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:28.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:28 smithi027 ceph-mon[36871]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:28.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-17T12:07:28.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:28.975 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:28 smithi141 ceph-mon[41225]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:28.975 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-17T12:07:28.975 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:29.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:28 smithi192 ceph-mon[41899]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:29.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-17T12:07:29.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:29.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:29 smithi141 ceph-mon[41225]: Deploying daemon osd.10 on smithi141 2023-12-17T12:07:29.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:29 smithi027 ceph-mon[36871]: Deploying daemon osd.10 on smithi141 2023-12-17T12:07:30.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:29 smithi192 ceph-mon[41899]: Deploying daemon osd.10 on smithi141 2023-12-17T12:07:30.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:30 smithi141 ceph-mon[41225]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:30.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:30 smithi027 ceph-mon[36871]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:31.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:30 smithi192 ceph-mon[41899]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:32.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:32 smithi141 ceph-mon[41225]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:32 smithi027 ceph-mon[36871]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:33.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:32 smithi192 ceph-mon[41899]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:34.230 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:34 smithi141 ceph-mon[41225]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:34.230 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:34.230 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:34.230 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:34.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:34 smithi192 ceph-mon[41899]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:34.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:34.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:34.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:34.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:34 smithi027 ceph-mon[36871]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:34.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:34.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:34.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.291 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.290+0000 7f196dffb700 1 -- 172.21.15.141:0/3032123483 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1970109730 con 0x7f19600a1500 2023-12-17T12:07:36.291 INFO:teuthology.orchestra.run.smithi141.stdout:Created osd(s) 10 on host 'smithi141' 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19600a1500 msgr2=0x7f19600a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19600a1500 0x7f19600a39c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c010 tx=0x7f195c005fb0 comp rx=0 tx=0).stop 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970101620 msgr2=0x7f19701996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970101620 0x7f19701996c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f196400d980 tx=0x7f196400dc90 comp rx=0 tx=0).stop 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 shutdown_connections 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19600a1500 0x7f19600a39c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19701031c0 0x7f19701a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1970101620 0x7f19701996c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 --2- 172.21.15.141:0/3032123483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1970102820 0x7f1970199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.292+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 >> 172.21.15.141:0/3032123483 conn(0x7f19700fcbe0 msgr2=0x7f1970108010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.293+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 shutdown_connections 2023-12-17T12:07:36.294 INFO:teuthology.orchestra.run.smithi141.stderr:2023-12-17T12:07:36.293+0000 7f1976ac6700 1 -- 172.21.15.141:0/3032123483 wait complete. 2023-12-17T12:07:36.491 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:36.491 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.491 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.491 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:36.491 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:36.491 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.492 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.492 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.492 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:36.673 DEBUG:teuthology.orchestra.run.smithi141:osd.10> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.10.service 2023-12-17T12:07:36.678 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2023-12-17T12:07:36.678 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-17T12:07:36.765 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:36.765 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.765 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.765 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:36.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:36.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:36.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:36.773 INFO:journalctl@ceph.osd.10.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:07:37.109 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:07:37.346 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:07:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:07:37.212+0000 7fc29c5f9640 -1 osd.10 0 log_to_monitors true 2023-12-17T12:07:37.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:37 smithi192 ceph-mon[41899]: from='osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-17T12:07:37.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:37 smithi192 ceph-mon[41899]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-17T12:07:37.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:37 smithi027 ceph-mon[36871]: from='osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-17T12:07:37.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:37 smithi027 ceph-mon[36871]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-17T12:07:37.692 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:37 smithi141 ceph-mon[41225]: from='osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-17T12:07:37.692 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:37 smithi141 ceph-mon[41225]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-17T12:07:38.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:38 smithi027 ceph-mon[36871]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:38.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:38 smithi027 ceph-mon[36871]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-17T12:07:38.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:38 smithi027 ceph-mon[36871]: from='osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:07:38.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:38 smithi027 ceph-mon[36871]: osdmap e67: 11 total, 10 up, 11 in 2023-12-17T12:07:38.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:38 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:38.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:38 smithi027 ceph-mon[36871]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:07:38.764 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:07:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:07:38.370+0000 7fc291c70700 -1 osd.10 0 waiting for initial osdmap 2023-12-17T12:07:38.764 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:07:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:07:38.386+0000 7fc28da6a700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:07:38.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:38 smithi141 ceph-mon[41225]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:38.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:38 smithi141 ceph-mon[41225]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-17T12:07:38.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:38 smithi141 ceph-mon[41225]: from='osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:07:38.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:38 smithi141 ceph-mon[41225]: osdmap e67: 11 total, 10 up, 11 in 2023-12-17T12:07:38.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:38 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:38.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:38 smithi141 ceph-mon[41225]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:07:38.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:38 smithi192 ceph-mon[41899]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:38.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:38 smithi192 ceph-mon[41899]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-17T12:07:38.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:38 smithi192 ceph-mon[41899]: from='osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:07:38.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:38 smithi192 ceph-mon[41899]: osdmap e67: 11 total, 10 up, 11 in 2023-12-17T12:07:38.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:38 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:38.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:38 smithi192 ceph-mon[41899]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]: dispatch 2023-12-17T12:07:38.902 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-17T12:07:38.902 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-17T12:07:38.902 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-12-17T12:07:38.902 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-12-17T12:07:38.902 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0304435 s, 344 MB/s 2023-12-17T12:07:38.902 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-12-17T12:07:39.319 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2023-12-17T12:07:39.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:39 smithi192 ceph-mon[41899]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:07:39.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:39 smithi192 ceph-mon[41899]: osdmap e68: 11 total, 10 up, 11 in 2023-12-17T12:07:39.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:39.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:39.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:39 smithi192 ceph-mon[41899]: osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] boot 2023-12-17T12:07:39.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:39 smithi192 ceph-mon[41899]: osdmap e69: 11 total, 11 up, 11 in 2023-12-17T12:07:39.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:39 smithi027 ceph-mon[36871]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:07:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:39 smithi027 ceph-mon[36871]: osdmap e68: 11 total, 10 up, 11 in 2023-12-17T12:07:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:39 smithi027 ceph-mon[36871]: osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] boot 2023-12-17T12:07:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:39 smithi027 ceph-mon[36871]: osdmap e69: 11 total, 11 up, 11 in 2023-12-17T12:07:39.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:39.731 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:07:39.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:39 smithi141 ceph-mon[41225]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi141", "root=default"]}]': finished 2023-12-17T12:07:39.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:39 smithi141 ceph-mon[41225]: osdmap e68: 11 total, 10 up, 11 in 2023-12-17T12:07:39.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:39.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:39.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:39 smithi141 ceph-mon[41225]: osd.10 [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] boot 2023-12-17T12:07:39.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:39 smithi141 ceph-mon[41225]: osdmap e69: 11 total, 11 up, 11 in 2023-12-17T12:07:39.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-17T12:07:40.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:07:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:07:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi141 2023-12-17T12:07:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi141 to 4317M 2023-12-17T12:07:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:40 smithi192 ceph-mon[41899]: osdmap e70: 11 total, 11 up, 11 in 2023-12-17T12:07:40.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:07:40.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:07:40.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:40.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi141 2023-12-17T12:07:40.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi141 to 4317M 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:40 smithi027 ceph-mon[36871]: osdmap e70: 11 total, 11 up, 11 in 2023-12-17T12:07:40.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:07:40.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi141 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi141 to 4317M 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:40.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:40.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:40.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:40 smithi141 ceph-mon[41225]: osdmap e70: 11 total, 11 up, 11 in 2023-12-17T12:07:40.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 -- 172.21.15.192:0/76233071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 msgr2=0x7f1710109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:40.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 --2- 172.21.15.192:0/76233071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 0x7f1710109a70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f170800b0a0 tx=0x7f170800b3b0 comp rx=0 tx=0).stop 2023-12-17T12:07:40.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 -- 172.21.15.192:0/76233071 shutdown_connections 2023-12-17T12:07:40.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 --2- 172.21.15.192:0/76233071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 0x7f1710109a70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:40.929 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 --2- 172.21.15.192:0/76233071 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1710101620 0x7f1710101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:40.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 --2- 172.21.15.192:0/76233071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1710102820 0x7f1710102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:40.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 -- 172.21.15.192:0/76233071 >> 172.21.15.192:0/76233071 conn(0x7f17100fcbe0 msgr2=0x7f17100ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:07:40.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 -- 172.21.15.192:0/76233071 shutdown_connections 2023-12-17T12:07:40.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.928+0000 7f1718082700 1 -- 172.21.15.192:0/76233071 wait complete. 2023-12-17T12:07:40.932 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.929+0000 7f1718082700 1 Processor -- start 2023-12-17T12:07:40.932 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.929+0000 7f1718082700 1 -- start start 2023-12-17T12:07:40.932 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.929+0000 7f1718082700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1710101620 0x7f1710075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:07:40.932 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.929+0000 7f1718082700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1710102820 0x7f17100739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f1718082700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 0x7f1710073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f1718082700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17100746a0 con 0x7f1710101620 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f1718082700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1710074810 con 0x7f1710102820 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f1718082700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17101a1600 con 0x7f17101031c0 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171661f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 0x7f1710073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171661f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 0x7f1710073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44156/0 (socket says 172.21.15.192:44156) 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171661f700 1 -- 172.21.15.192:0/3428237181 learned_addr learned my addr 172.21.15.192:0/3428237181 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171561d700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1710102820 0x7f17100739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f1715e1e700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1710101620 0x7f1710075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171661f700 1 -- 172.21.15.192:0/3428237181 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1710102820 msgr2=0x7f17100739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171661f700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1710102820 0x7f17100739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171661f700 1 -- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1710101620 msgr2=0x7f1710075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:40.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171661f700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1710101620 0x7f1710075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:40.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.930+0000 7f171661f700 1 -- 172.21.15.192:0/3428237181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708009d00 con 0x7f17101031c0 2023-12-17T12:07:40.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.931+0000 7f171661f700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 0x7f1710073f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1708007a60 tx=0x7f1708004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:07:40.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.931+0000 7f1706ffd700 1 -- 172.21.15.192:0/3428237181 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170800e040 con 0x7f17101031c0 2023-12-17T12:07:40.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.931+0000 7f1706ffd700 1 -- 172.21.15.192:0/3428237181 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17080056e0 con 0x7f17101031c0 2023-12-17T12:07:40.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.931+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17101a1830 con 0x7f17101031c0 2023-12-17T12:07:40.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.931+0000 7f1706ffd700 1 -- 172.21.15.192:0/3428237181 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170801d9b0 con 0x7f17101031c0 2023-12-17T12:07:40.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.931+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17101a1c40 con 0x7f17101031c0 2023-12-17T12:07:40.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.932+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1710108aa0 con 0x7f17101031c0 2023-12-17T12:07:40.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.934+0000 7f1706ffd700 1 -- 172.21.15.192:0/3428237181 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1708005850 con 0x7f17101031c0 2023-12-17T12:07:40.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.935+0000 7f1706ffd700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:07:40.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.935+0000 7f1706ffd700 1 -- 172.21.15.192:0/3428237181 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f17080ce200 con 0x7f17101031c0 2023-12-17T12:07:40.937 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.936+0000 7f1715e1e700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:07:40.937 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.937+0000 7f1715e1e700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16fc0a1480 0x7f16fc0a3940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1710102680 tx=0x7f170c00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:07:40.938 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:40.937+0000 7f1706ffd700 1 -- 172.21.15.192:0/3428237181 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1708097e80 con 0x7f17101031c0 2023-12-17T12:07:41.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:41.146+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f171002d380 con 0x7f16fc0a1480 2023-12-17T12:07:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:41 smithi192 ceph-mon[41899]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:41 smithi192 ceph-mon[41899]: osdmap e71: 11 total, 11 up, 11 in 2023-12-17T12:07:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:41 smithi192 ceph-mon[41899]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:07:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:41 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:07:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:41 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:07:42.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:41 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:41 smithi027 ceph-mon[36871]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:41 smithi027 ceph-mon[36871]: osdmap e71: 11 total, 11 up, 11 in 2023-12-17T12:07:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:41 smithi027 ceph-mon[36871]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:07:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:41 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:07:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:41 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:07:42.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:41 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:42.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:41 smithi141 ceph-mon[41225]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:42.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:41 smithi141 ceph-mon[41225]: osdmap e71: 11 total, 11 up, 11 in 2023-12-17T12:07:42.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:41 smithi141 ceph-mon[41225]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:07:42.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:41 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-17T12:07:42.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:41 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-17T12:07:42.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:41 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:43.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:42 smithi027 ceph-mon[36871]: osdmap e72: 11 total, 11 up, 11 in 2023-12-17T12:07:43.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:42 smithi141 ceph-mon[41225]: osdmap e72: 11 total, 11 up, 11 in 2023-12-17T12:07:43.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:42 smithi192 ceph-mon[41899]: osdmap e72: 11 total, 11 up, 11 in 2023-12-17T12:07:44.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:43 smithi027 ceph-mon[36871]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-17T12:07:44.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:43 smithi027 ceph-mon[36871]: osdmap e73: 11 total, 11 up, 11 in 2023-12-17T12:07:44.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:43 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/1276663103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]: dispatch 2023-12-17T12:07:44.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:43 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]: dispatch 2023-12-17T12:07:44.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:43 smithi027 ceph-mon[36871]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]': finished 2023-12-17T12:07:44.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:43 smithi027 ceph-mon[36871]: osdmap e74: 12 total, 11 up, 12 in 2023-12-17T12:07:44.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:43 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:07:44.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:43 smithi141 ceph-mon[41225]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-17T12:07:44.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:43 smithi141 ceph-mon[41225]: osdmap e73: 11 total, 11 up, 11 in 2023-12-17T12:07:44.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:43 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/1276663103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]: dispatch 2023-12-17T12:07:44.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:43 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]: dispatch 2023-12-17T12:07:44.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:43 smithi141 ceph-mon[41225]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]': finished 2023-12-17T12:07:44.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:43 smithi141 ceph-mon[41225]: osdmap e74: 12 total, 11 up, 12 in 2023-12-17T12:07:44.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:43 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:07:44.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:43 smithi192 ceph-mon[41899]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-17T12:07:44.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:43 smithi192 ceph-mon[41899]: osdmap e73: 11 total, 11 up, 11 in 2023-12-17T12:07:44.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:43 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/1276663103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]: dispatch 2023-12-17T12:07:44.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:43 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]: dispatch 2023-12-17T12:07:44.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:43 smithi192 ceph-mon[41899]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9e69f0d-0fe3-4559-93ac-a66f20c0ce15"}]': finished 2023-12-17T12:07:44.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:43 smithi192 ceph-mon[41899]: osdmap e74: 12 total, 11 up, 12 in 2023-12-17T12:07:44.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:43 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:07:45.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:44 smithi027 ceph-mon[36871]: from='client.? 172.21.15.192:0/394723424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:45.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:44 smithi141 ceph-mon[41225]: from='client.? 172.21.15.192:0/394723424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:45.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:44 smithi192 ceph-mon[41899]: from='client.? 172.21.15.192:0/394723424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-17T12:07:46.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:45 smithi027 ceph-mon[36871]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-17T12:07:46.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:45 smithi141 ceph-mon[41225]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-17T12:07:46.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:45 smithi192 ceph-mon[41899]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-17T12:07:48.041 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:47 smithi192 ceph-mon[41899]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-12-17T12:07:48.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:47 smithi027 ceph-mon[36871]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-12-17T12:07:48.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:47 smithi141 ceph-mon[41225]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-12-17T12:07:50.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:49 smithi027 ceph-mon[36871]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-12-17T12:07:50.265 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:49 smithi141 ceph-mon[41225]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-12-17T12:07:50.265 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:49 smithi192 ceph-mon[41899]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-12-17T12:07:51.218 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-17T12:07:51.218 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:51.218 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:50 smithi192 ceph-mon[41899]: Deploying daemon osd.11 on smithi192 2023-12-17T12:07:51.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-17T12:07:51.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:51.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:50 smithi141 ceph-mon[41225]: Deploying daemon osd.11 on smithi192 2023-12-17T12:07:51.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-17T12:07:51.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:51.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:50 smithi027 ceph-mon[36871]: Deploying daemon osd.11 on smithi192 2023-12-17T12:07:52.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:51 smithi141 ceph-mon[41225]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:52.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:51 smithi192 ceph-mon[41899]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:52.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:51 smithi027 ceph-mon[36871]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:54.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:53 smithi141 ceph-mon[41225]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:54.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:53 smithi192 ceph-mon[41899]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:54.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:53 smithi027 ceph-mon[36871]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:56.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:55 smithi192 ceph-mon[41899]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:56.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:56.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:56.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:55 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:55 smithi141 ceph-mon[41225]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:56.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:55 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:56.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:55 smithi027 ceph-mon[36871]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:56.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:56.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:56.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:55 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:58.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:57 smithi192 ceph-mon[41899]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:58.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:57 smithi141 ceph-mon[41225]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:58.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:57 smithi027 ceph-mon[36871]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:07:58.968 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.967+0000 7f1706ffd700 1 -- 172.21.15.192:0/3428237181 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f171002d380 con 0x7f16fc0a1480 2023-12-17T12:07:58.968 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16fc0a1480 msgr2=0x7f16fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16fc0a1480 0x7f16fc0a3940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1710102680 tx=0x7f170c00b410 comp rx=0 tx=0).stop 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 msgr2=0x7f1710073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 0x7f1710073f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1708007a60 tx=0x7f1708004c80 comp rx=0 tx=0).stop 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 shutdown_connections 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17101031c0 0x7f1710073f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1710101620 0x7f1710075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:58.971 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 --2- 172.21.15.192:0/3428237181 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1710102820 0x7f17100739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:07:58.972 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 >> 172.21.15.192:0/3428237181 conn(0x7f17100fcbe0 msgr2=0x7f1710105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:07:58.972 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 shutdown_connections 2023-12-17T12:07:58.972 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:07:58.970+0000 7f1718082700 1 -- 172.21.15.192:0/3428237181 wait complete. 2023-12-17T12:07:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:59.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:59.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:59.268 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.269 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:07:59 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:59.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:07:59.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:07:59.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:07:59.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:07:59 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:07:59.417 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.11.service 2023-12-17T12:07:59.421 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-12-17T12:07:59.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd stat -f json 2023-12-17T12:07:59.505 INFO:journalctl@ceph.osd.11.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:07:59.876 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:08:00.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:00 smithi192 ceph-mon[41899]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:00.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:00 smithi192 ceph-mon[41899]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-17T12:08:00.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:07:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:07:59.789+0000 7fe69201a640 -1 osd.11 0 log_to_monitors true 2023-12-17T12:08:00.400 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:00 smithi027 ceph-mon[36871]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:00.400 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:00 smithi027 ceph-mon[36871]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-17T12:08:00.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:00 smithi141 ceph-mon[41225]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:00.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:00 smithi141 ceph-mon[41225]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 -- 172.21.15.27:0/328366282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c102820 msgr2=0x7f361c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 --2- 172.21.15.27:0/328366282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c102820 0x7f361c102c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f360c0099c0 tx=0x7f360c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 -- 172.21.15.27:0/328366282 shutdown_connections 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 --2- 172.21.15.27:0/328366282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f361c101620 0x7f361c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 --2- 172.21.15.27:0/328366282 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f361c1031c0 0x7f361c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 --2- 172.21.15.27:0/328366282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c102820 0x7f361c102c80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 -- 172.21.15.27:0/328366282 >> 172.21.15.27:0/328366282 conn(0x7f361c0fcbe0 msgr2=0x7f361c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 -- 172.21.15.27:0/328366282 shutdown_connections 2023-12-17T12:08:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.139+0000 7f3624390700 1 -- 172.21.15.27:0/328366282 wait complete. 2023-12-17T12:08:01.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.141+0000 7f3624390700 1 Processor -- start 2023-12-17T12:08:01.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.141+0000 7f3624390700 1 -- start start 2023-12-17T12:08:01.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.141+0000 7f3624390700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f361c101620 0x7f361c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.141+0000 7f3624390700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f361c102820 0x7f361c199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.141+0000 7f3624390700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c1031c0 0x7f361c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.141+0000 7f3624390700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1a1190 con 0x7f361c1031c0 2023-12-17T12:08:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.141+0000 7f3624390700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1a12d0 con 0x7f361c101620 2023-12-17T12:08:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.141+0000 7f3624390700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1a1460 con 0x7f361c102820 2023-12-17T12:08:01.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c1031c0 0x7f361c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:01.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362192b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f361c102820 0x7f361c199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:01.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c1031c0 0x7f361c1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39290/0 (socket says 172.21.15.27:39290) 2023-12-17T12:08:01.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 -- 172.21.15.27:0/3715405765 learned_addr learned my addr 172.21.15.27:0/3715405765 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:01.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362212c700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f361c101620 0x7f361c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:01.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 -- 172.21.15.27:0/3715405765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f361c102820 msgr2=0x7f361c199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:01.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f361c102820 0x7f361c199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 -- 172.21.15.27:0/3715405765 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f361c101620 msgr2=0x7f361c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f361c101620 0x7f361c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 -- 172.21.15.27:0/3715405765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618009520 con 0x7f361c1031c0 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.142+0000 7f362292d700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c1031c0 0x7f361c1a0bc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3614007f10 tx=0x7f361400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.143+0000 7f36137fe700 1 -- 172.21.15.27:0/3715405765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614017070 con 0x7f361c1031c0 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.143+0000 7f36137fe700 1 -- 172.21.15.27:0/3715405765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f361400f040 con 0x7f361c1031c0 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.143+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c0095f0 con 0x7f361c1031c0 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.143+0000 7f36137fe700 1 -- 172.21.15.27:0/3715405765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614005570 con 0x7f361c1031c0 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.143+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c1a1ab0 con 0x7f361c1031c0 2023-12-17T12:08:01.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.144+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c04f290 con 0x7f361c1031c0 2023-12-17T12:08:01.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.146+0000 7f36137fe700 1 -- 172.21.15.27:0/3715405765 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f361400db20 con 0x7f361c1031c0 2023-12-17T12:08:01.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.147+0000 7f36137fe700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36080a1500 0x7f36080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:01.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.147+0000 7f36137fe700 1 -- 172.21.15.27:0/3715405765 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f36140cbef0 con 0x7f361c1031c0 2023-12-17T12:08:01.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.147+0000 7f362212c700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36080a1500 0x7f36080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:01.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.148+0000 7f362212c700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36080a1500 0x7f36080a39c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f361c102680 tx=0x7f3618009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:01.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.152+0000 7f36137fe700 1 -- 172.21.15.27:0/3715405765 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3614095a70 con 0x7f361c1031c0 2023-12-17T12:08:01.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:01 smithi027 ceph-mon[36871]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-17T12:08:01.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:01 smithi027 ceph-mon[36871]: osdmap e75: 12 total, 11 up, 12 in 2023-12-17T12:08:01.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:01 smithi027 ceph-mon[36871]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:08:01.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:01 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:01.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:01 smithi027 ceph-mon[36871]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:08:01.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:01 smithi027 ceph-mon[36871]: osdmap e76: 12 total, 11 up, 12 in 2023-12-17T12:08:01.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:01 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-mon[41899]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-17T12:08:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-mon[41899]: osdmap e75: 12 total, 11 up, 12 in 2023-12-17T12:08:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-mon[41899]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:08:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-mon[41899]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:08:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-mon[41899]: osdmap e76: 12 total, 11 up, 12 in 2023-12-17T12:08:01.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:01.312 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:08:01.049+0000 7fe687e92700 -1 osd.11 0 waiting for initial osdmap 2023-12-17T12:08:01.312 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:08:01 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:08:01.065+0000 7fe681487700 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-17T12:08:01.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.362+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f361c10cce0 con 0x7f361c1031c0 2023-12-17T12:08:01.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.363+0000 7f36137fe700 1 -- 172.21.15.27:0/3715405765 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f3614099090 con 0x7f361c1031c0 2023-12-17T12:08:01.364 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:01.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36080a1500 msgr2=0x7f36080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36080a1500 0x7f36080a39c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f361c102680 tx=0x7f3618009260 comp rx=0 tx=0).stop 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c1031c0 msgr2=0x7f361c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c1031c0 0x7f361c1a0bc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3614007f10 tx=0x7f361400d3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 shutdown_connections 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36080a1500 0x7f36080a39c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f361c1031c0 0x7f361c1a0bc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f361c101620 0x7f361c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 --2- 172.21.15.27:0/3715405765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f361c102820 0x7f361c199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 >> 172.21.15.27:0/3715405765 conn(0x7f361c0fcbe0 msgr2=0x7f361c108060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 shutdown_connections 2023-12-17T12:08:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:01.365+0000 7f3624390700 1 -- 172.21.15.27:0/3715405765 wait complete. 2023-12-17T12:08:01.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:01 smithi141 ceph-mon[41225]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-17T12:08:01.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:01 smithi141 ceph-mon[41225]: osdmap e75: 12 total, 11 up, 12 in 2023-12-17T12:08:01.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:01 smithi141 ceph-mon[41225]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-12-17T12:08:01.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:01 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:01.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:01 smithi141 ceph-mon[41225]: from='osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-12-17T12:08:01.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:01 smithi141 ceph-mon[41225]: osdmap e76: 12 total, 11 up, 12 in 2023-12-17T12:08:01.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:01 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:01.758 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1702814858,"num_in_osds":12,"osd_in_since":1702814863,"num_remapped_pgs":0} 2023-12-17T12:08:02.306 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:02 smithi192 ceph-mon[41899]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:02.306 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:02 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:02.306 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:02 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3715405765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-17T12:08:02.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:02 smithi027 ceph-mon[36871]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:02.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:02 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:02.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:02 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3715405765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-17T12:08:02.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:02 smithi141 ceph-mon[41225]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:02.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:02 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:02.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:02 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3715405765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-17T12:08:02.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd stat -f json 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: purged_snaps scrub starts 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: purged_snaps scrub ok 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] boot 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: osdmap e77: 12 total, 12 up, 12 in 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: Detected new or changed devices on smithi192 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: Adjusting osd_memory_target on smithi192 to 4317M 2023-12-17T12:08:03.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:03.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:03.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:03 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.185 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:08:03.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: purged_snaps scrub starts 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: purged_snaps scrub ok 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] boot 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: osdmap e77: 12 total, 12 up, 12 in 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: Detected new or changed devices on smithi192 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: Adjusting osd_memory_target on smithi192 to 4317M 2023-12-17T12:08:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:03.312 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:03 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: purged_snaps scrub starts 2023-12-17T12:08:03.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: purged_snaps scrub ok 2023-12-17T12:08:03.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:03.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: osd.11 [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] boot 2023-12-17T12:08:03.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: osdmap e77: 12 total, 12 up, 12 in 2023-12-17T12:08:03.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: Detected new or changed devices on smithi192 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: Adjusting osd_memory_target on smithi192 to 4317M 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:03.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:03 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:04.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.344+0000 7fd717c3b700 1 -- 172.21.15.27:0/1395497094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 msgr2=0x7fd710101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:04.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.344+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1395497094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 0x7fd710101a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0099c0 tx=0x7fd70c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.345+0000 7fd717c3b700 1 -- 172.21.15.27:0/1395497094 shutdown_connections 2023-12-17T12:08:04.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.345+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1395497094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7101031c0 0x7fd710109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.345+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1395497094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 0x7fd710101a40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.345+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1395497094 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd710102820 0x7fd710102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.345+0000 7fd717c3b700 1 -- 172.21.15.27:0/1395497094 >> 172.21.15.27:0/1395497094 conn(0x7fd7100fcbe0 msgr2=0x7fd7100ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:04.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.345+0000 7fd717c3b700 1 -- 172.21.15.27:0/1395497094 shutdown_connections 2023-12-17T12:08:04.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.345+0000 7fd717c3b700 1 -- 172.21.15.27:0/1395497094 wait complete. 2023-12-17T12:08:04.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.346+0000 7fd717c3b700 1 Processor -- start 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.346+0000 7fd717c3b700 1 -- start start 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.346+0000 7fd717c3b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 0x7fd7101996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.346+0000 7fd717c3b700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd710102820 0x7fd710199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.346+0000 7fd717c3b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7101031c0 0x7fd7101a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.346+0000 7fd717c3b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7101a11c0 con 0x7fd710101620 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.346+0000 7fd717c3b700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7101a1300 con 0x7fd710102820 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.346+0000 7fd717c3b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7101a1490 con 0x7fd7101031c0 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7159d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 0x7fd7101996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7159d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 0x7fd7101996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39320/0 (socket says 172.21.15.27:39320) 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7159d7700 1 -- 172.21.15.27:0/1146014688 learned_addr learned my addr 172.21.15.27:0/1146014688 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:04.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7151d6700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd710102820 0x7fd710199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7161d8700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7101031c0 0x7fd7101a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7159d7700 1 -- 172.21.15.27:0/1146014688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7101031c0 msgr2=0x7fd7101a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7159d7700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7101031c0 0x7fd7101a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7159d7700 1 -- 172.21.15.27:0/1146014688 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd710102820 msgr2=0x7fd710199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7159d7700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd710102820 0x7fd710199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7159d7700 1 -- 172.21.15.27:0/1146014688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c0095f0 con 0x7fd710101620 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.347+0000 7fd7161d8700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7101031c0 0x7fd7101a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.348+0000 7fd7159d7700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 0x7fd7101996d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098e0 tx=0x7fd70c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.348+0000 7fd706ffd700 1 -- 172.21.15.27:0/1146014688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c01d070 con 0x7fd710101620 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.348+0000 7fd706ffd700 1 -- 172.21.15.27:0/1146014688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd70c00bd10 con 0x7fd710101620 2023-12-17T12:08:04.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.348+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7101a1710 con 0x7fd710101620 2023-12-17T12:08:04.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.348+0000 7fd706ffd700 1 -- 172.21.15.27:0/1146014688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c004bd0 con 0x7fd710101620 2023-12-17T12:08:04.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.348+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7101a1b80 con 0x7fd710101620 2023-12-17T12:08:04.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.349+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71004f290 con 0x7fd710101620 2023-12-17T12:08:04.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.351+0000 7fd706ffd700 1 -- 172.21.15.27:0/1146014688 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd70c004d30 con 0x7fd710101620 2023-12-17T12:08:04.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.354+0000 7fd706ffd700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd6fc0a1500 0x7fd6fc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:04.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.354+0000 7fd706ffd700 1 -- 172.21.15.27:0/1146014688 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd70c0cda30 con 0x7fd710101620 2023-12-17T12:08:04.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.354+0000 7fd7151d6700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd6fc0a1500 0x7fd6fc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:04.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.354+0000 7fd7151d6700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd6fc0a1500 0x7fd6fc0a39c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd700009c60 tx=0x7fd700009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:04.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.356+0000 7fd706ffd700 1 -- 172.21.15.27:0/1146014688 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd70c097520 con 0x7fd710101620 2023-12-17T12:08:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.570+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd71010cd30 con 0x7fd710101620 2023-12-17T12:08:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.570+0000 7fd706ffd700 1 -- 172.21.15.27:0/1146014688 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7fd70c09ab40 con 0x7fd710101620 2023-12-17T12:08:04.572 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:04.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd6fc0a1500 msgr2=0x7fd6fc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:04.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd6fc0a1500 0x7fd6fc0a39c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd700009c60 tx=0x7fd700009260 comp rx=0 tx=0).stop 2023-12-17T12:08:04.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 msgr2=0x7fd7101996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:04.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 0x7fd7101996d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098e0 tx=0x7fd70c00fa00 comp rx=0 tx=0).stop 2023-12-17T12:08:04.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 shutdown_connections 2023-12-17T12:08:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd6fc0a1500 0x7fd6fc0a39c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7101031c0 0x7fd7101a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710101620 0x7fd7101996d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 --2- 172.21.15.27:0/1146014688 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd710102820 0x7fd710199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 >> 172.21.15.27:0/1146014688 conn(0x7fd7100fcbe0 msgr2=0x7fd710107fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 shutdown_connections 2023-12-17T12:08:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:04.573+0000 7fd717c3b700 1 -- 172.21.15.27:0/1146014688 wait complete. 2023-12-17T12:08:04.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:04 smithi027 ceph-mon[36871]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:04.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:04 smithi027 ceph-mon[36871]: osdmap e78: 12 total, 12 up, 12 in 2023-12-17T12:08:04.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:04 smithi141 ceph-mon[41225]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:04.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:04 smithi141 ceph-mon[41225]: osdmap e78: 12 total, 12 up, 12 in 2023-12-17T12:08:04.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:04 smithi192 ceph-mon[41899]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-17T12:08:04.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:04 smithi192 ceph-mon[41899]: osdmap e78: 12 total, 12 up, 12 in 2023-12-17T12:08:04.978 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1702814882,"num_in_osds":12,"osd_in_since":1702814863,"num_remapped_pgs":0} 2023-12-17T12:08:04.979 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-17T12:08:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:08:05.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:05 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1146014688' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-17T12:08:05.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:05 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1146014688' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-17T12:08:05.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:05 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1146014688' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-17T12:08:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 -- 172.21.15.27:0/2371305668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4941076f0 msgr2=0x7fb494107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 --2- 172.21.15.27:0/2371305668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4941076f0 0x7fb494107ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb488009990 tx=0x7fb488009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 -- 172.21.15.27:0/2371305668 shutdown_connections 2023-12-17T12:08:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 --2- 172.21.15.27:0/2371305668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb494102040 0x7fb49410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 --2- 172.21.15.27:0/2371305668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4941076f0 0x7fb494107ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 --2- 172.21.15.27:0/2371305668 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb494101680 0x7fb494101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 -- 172.21.15.27:0/2371305668 >> 172.21.15.27:0/2371305668 conn(0x7fb4940787c0 msgr2=0x7fb494078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 -- 172.21.15.27:0/2371305668 shutdown_connections 2023-12-17T12:08:06.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.653+0000 7fb49ada1700 1 -- 172.21.15.27:0/2371305668 wait complete. 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.654+0000 7fb49ada1700 1 Processor -- start 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.654+0000 7fb49ada1700 1 -- start start 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb49ada1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb494101680 0x7fb49419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb49ada1700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb494102040 0x7fb49419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb49ada1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4941076f0 0x7fb4941976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb49ada1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb494197d20 con 0x7fb494101680 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb49ada1700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb494197e90 con 0x7fb494102040 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb49ada1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb494198050 con 0x7fb4941076f0 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb498b3d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb494101680 0x7fb49419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb498b3d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb494101680 0x7fb49419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33124/0 (socket says 172.21.15.27:33124) 2023-12-17T12:08:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb498b3d700 1 -- 172.21.15.27:0/1532388459 learned_addr learned my addr 172.21.15.27:0/1532388459 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb493fff700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb494102040 0x7fb49419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb49933e700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4941076f0 0x7fb4941976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb498b3d700 1 -- 172.21.15.27:0/1532388459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4941076f0 msgr2=0x7fb4941976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb498b3d700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4941076f0 0x7fb4941976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb498b3d700 1 -- 172.21.15.27:0/1532388459 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb494102040 msgr2=0x7fb49419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb498b3d700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb494102040 0x7fb49419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb498b3d700 1 -- 172.21.15.27:0/1532388459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4880095f0 con 0x7fb494101680 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.655+0000 7fb49933e700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4941076f0 0x7fb4941976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.656+0000 7fb498b3d700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb494101680 0x7fb49419d580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb488006010 tx=0x7fb488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.656+0000 7fb491ffb700 1 -- 172.21.15.27:0/1532388459 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48801d070 con 0x7fb494101680 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.656+0000 7fb491ffb700 1 -- 172.21.15.27:0/1532388459 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb48800fc00 con 0x7fb494101680 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.656+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4941982d0 con 0x7fb494101680 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.656+0000 7fb491ffb700 1 -- 172.21.15.27:0/1532388459 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488022b90 con 0x7fb494101680 2023-12-17T12:08:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.656+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4941a4570 con 0x7fb494101680 2023-12-17T12:08:06.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.657+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb494109b80 con 0x7fb494101680 2023-12-17T12:08:06.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.659+0000 7fb491ffb700 1 -- 172.21.15.27:0/1532388459 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb48800f770 con 0x7fb494101680 2023-12-17T12:08:06.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:06 smithi027 ceph-mon[36871]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:06.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.661+0000 7fb491ffb700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fb4840a1500 0x7fb4840a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:06.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.661+0000 7fb491ffb700 1 -- 172.21.15.27:0/1532388459 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb4880cdd20 con 0x7fb494101680 2023-12-17T12:08:06.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.661+0000 7fb493fff700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fb4840a1500 0x7fb4840a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:06.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.662+0000 7fb493fff700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fb4840a1500 0x7fb4840a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb494198dd0 tx=0x7fb48000b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.665+0000 7fb491ffb700 1 -- 172.21.15.27:0/1532388459 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb488097810 con 0x7fb494101680 2023-12-17T12:08:06.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:06 smithi141 ceph-mon[41225]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:06.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:06 smithi192 ceph-mon[41899]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:06.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.875+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb494102d10 con 0x7fb494101680 2023-12-17T12:08:06.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.876+0000 7fb491ffb700 1 -- 172.21.15.27:0/1532388459 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18433 (secure 0 0 0) 0x7fb48809ae30 con 0x7fb494101680 2023-12-17T12:08:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:06.878 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":78,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:08:03.382807+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fb4840a1500 msgr2=0x7fb4840a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fb4840a1500 0x7fb4840a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb494198dd0 tx=0x7fb48000b410 comp rx=0 tx=0).stop 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb494101680 msgr2=0x7fb49419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb494101680 0x7fb49419d580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb488006010 tx=0x7fb488004730 comp rx=0 tx=0).stop 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 shutdown_connections 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fb4840a1500 0x7fb4840a39c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4941076f0 0x7fb4941976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb494101680 0x7fb49419d580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 --2- 172.21.15.27:0/1532388459 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb494102040 0x7fb49419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:06.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 >> 172.21.15.27:0/1532388459 conn(0x7fb4940787c0 msgr2=0x7fb494100340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:06.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 shutdown_connections 2023-12-17T12:08:06.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:06.878+0000 7fb49ada1700 1 -- 172.21.15.27:0/1532388459 wait complete. 2023-12-17T12:08:07.398 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-17T12:04:41.530587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-17T12:08:07.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-17T12:08:07.467 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:07 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1532388459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:07.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:07 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1532388459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:07.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:07 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1532388459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:08:08.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:08 smithi027 ceph-mon[36871]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:08.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:08 smithi141 ceph-mon[41225]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:08.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:08 smithi192 ceph-mon[41899]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:09.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 -- 172.21.15.27:0/3471244251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4194102040 msgr2=0x7f419410e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:09.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 --2- 172.21.15.27:0/3471244251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4194102040 0x7f419410e430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f418c00b0a0 tx=0x7f418c00b3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 -- 172.21.15.27:0/3471244251 shutdown_connections 2023-12-17T12:08:09.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 --2- 172.21.15.27:0/3471244251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4194102040 0x7f419410e430 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 --2- 172.21.15.27:0/3471244251 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f41941076f0 0x7f4194107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 --2- 172.21.15.27:0/3471244251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4194101680 0x7f4194101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 -- 172.21.15.27:0/3471244251 >> 172.21.15.27:0/3471244251 conn(0x7f41940787c0 msgr2=0x7f4194078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 -- 172.21.15.27:0/3471244251 shutdown_connections 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.048+0000 7f419a518700 1 -- 172.21.15.27:0/3471244251 wait complete. 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.049+0000 7f419a518700 1 Processor -- start 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.049+0000 7f419a518700 1 -- start start 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f419a518700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4194101680 0x7f41940738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f419a518700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4194102040 0x7f4194073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f419a518700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41941076f0 0x7f4194074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f419a518700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a5e60 con 0x7f41941076f0 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f419a518700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a5fd0 con 0x7f4194101680 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f419a518700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a6190 con 0x7f4194102040 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4198ab5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41941076f0 0x7f4194074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4198ab5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41941076f0 0x7f4194074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33142/0 (socket says 172.21.15.27:33142) 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4198ab5700 1 -- 172.21.15.27:0/249188174 learned_addr learned my addr 172.21.15.27:0/249188174 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f41937fe700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4194102040 0x7f4194073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4193fff700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4194101680 0x7f41940738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4198ab5700 1 -- 172.21.15.27:0/249188174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4194102040 msgr2=0x7f4194073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4198ab5700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4194102040 0x7f4194073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4198ab5700 1 -- 172.21.15.27:0/249188174 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4194101680 msgr2=0x7f41940738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4198ab5700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4194101680 0x7f41940738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4198ab5700 1 -- 172.21.15.27:0/249188174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c009d00 con 0x7f41941076f0 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.050+0000 7f4193fff700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4194101680 0x7f41940738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.051+0000 7f4198ab5700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41941076f0 0x7f4194074350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f418c007930 tx=0x7f418c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.051+0000 7f41917fa700 1 -- 172.21.15.27:0/249188174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00e040 con 0x7f41941076f0 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.051+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41941a6410 con 0x7f41941076f0 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.051+0000 7f41917fa700 1 -- 172.21.15.27:0/249188174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f418c004c50 con 0x7f41941076f0 2023-12-17T12:08:09.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.051+0000 7f41917fa700 1 -- 172.21.15.27:0/249188174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0052a0 con 0x7f41941076f0 2023-12-17T12:08:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.051+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41941a6830 con 0x7f41941076f0 2023-12-17T12:08:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.052+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4194109b80 con 0x7f41941076f0 2023-12-17T12:08:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.054+0000 7f41917fa700 1 -- 172.21.15.27:0/249188174 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f418c01b970 con 0x7f41941076f0 2023-12-17T12:08:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.055+0000 7f41917fa700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f417c0a1480 0x7f417c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.056+0000 7f41917fa700 1 -- 172.21.15.27:0/249188174 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f418c0ce140 con 0x7f41941076f0 2023-12-17T12:08:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.056+0000 7f4193fff700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f417c0a1480 0x7f417c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.056+0000 7f4193fff700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f417c0a1480 0x7f417c0a3940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4184009b50 tx=0x7f4184009550 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:09.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.060+0000 7f41917fa700 1 -- 172.21.15.27:0/249188174 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f418c097c30 con 0x7f41941076f0 2023-12-17T12:08:09.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.271+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4194074db0 con 0x7f41941076f0 2023-12-17T12:08:09.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.272+0000 7f41917fa700 1 -- 172.21.15.27:0/249188174 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f418c09b250 con 0x7f41941076f0 2023-12-17T12:08:09.273 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f417c0a1480 msgr2=0x7f417c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f417c0a1480 0x7f417c0a3940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4184009b50 tx=0x7f4184009550 comp rx=0 tx=0).stop 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41941076f0 msgr2=0x7f4194074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41941076f0 0x7f4194074350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f418c007930 tx=0x7f418c007a10 comp rx=0 tx=0).stop 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 shutdown_connections 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f417c0a1480 0x7f417c0a3940 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41941076f0 0x7f4194074350 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4194101680 0x7f41940738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 --2- 172.21.15.27:0/249188174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4194102040 0x7f4194073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 >> 172.21.15.27:0/249188174 conn(0x7f41940787c0 msgr2=0x7f4194100370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:09.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 shutdown_connections 2023-12-17T12:08:09.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:09.274+0000 7f419a518700 1 -- 172.21.15.27:0/249188174 wait complete. 2023-12-17T12:08:09.660 INFO:tasks.cephadm:Adding mds.a on smithi027 2023-12-17T12:08:09.660 INFO:tasks.cephadm:Adding mds.d on smithi027 2023-12-17T12:08:09.660 INFO:tasks.cephadm:Adding mds.g on smithi027 2023-12-17T12:08:09.660 INFO:tasks.cephadm:Adding mds.j on smithi027 2023-12-17T12:08:09.660 INFO:tasks.cephadm:Adding mds.b on smithi141 2023-12-17T12:08:09.660 INFO:tasks.cephadm:Adding mds.e on smithi141 2023-12-17T12:08:09.660 INFO:tasks.cephadm:Adding mds.h on smithi141 2023-12-17T12:08:09.660 INFO:tasks.cephadm:Adding mds.k on smithi141 2023-12-17T12:08:09.661 INFO:tasks.cephadm:Adding mds.c on smithi192 2023-12-17T12:08:09.661 INFO:tasks.cephadm:Adding mds.f on smithi192 2023-12-17T12:08:09.661 INFO:tasks.cephadm:Adding mds.i on smithi192 2023-12-17T12:08:09.661 INFO:tasks.cephadm:Adding mds.l on smithi192 2023-12-17T12:08:09.661 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph orch apply mds all '12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2023-12-17T12:08:09.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:09 smithi027 ceph-mon[36871]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:09 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/249188174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:10.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:09 smithi192 ceph-mon[41899]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:09 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/249188174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:10.080 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.c/config 2023-12-17T12:08:10.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:09 smithi141 ceph-mon[41225]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:10.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:09 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/249188174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.380+0000 7f473efa4700 1 -- 172.21.15.192:0/1552341733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738075600 msgr2=0x7f4738073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.380+0000 7f473efa4700 1 --2- 172.21.15.192:0/1552341733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738075600 0x7f4738073a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f472c009990 tx=0x7f472c009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.380+0000 7f473efa4700 1 -- 172.21.15.192:0/1552341733 shutdown_connections 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.380+0000 7f473efa4700 1 --2- 172.21.15.192:0/1552341733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738075600 0x7f4738073a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.380+0000 7f473efa4700 1 --2- 172.21.15.192:0/1552341733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f473810dc00 0x7f473810fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.380+0000 7f473efa4700 1 --2- 172.21.15.192:0/1552341733 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4738074030 0x7f47380744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.380+0000 7f473efa4700 1 -- 172.21.15.192:0/1552341733 >> 172.21.15.192:0/1552341733 conn(0x7f47380fcbe0 msgr2=0x7f47380ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.381+0000 7f473efa4700 1 -- 172.21.15.192:0/1552341733 shutdown_connections 2023-12-17T12:08:11.382 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.381+0000 7f473efa4700 1 -- 172.21.15.192:0/1552341733 wait complete. 2023-12-17T12:08:11.384 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.381+0000 7f473efa4700 1 Processor -- start 2023-12-17T12:08:11.384 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473efa4700 1 -- start start 2023-12-17T12:08:11.384 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473efa4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738074030 0x7f473819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:11.384 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473efa4700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4738075600 0x7f473819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473efa4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f473810dc00 0x7f47381a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473efa4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381a55b0 con 0x7f473810dc00 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473efa4700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381a56f0 con 0x7f4738075600 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473efa4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381a5880 con 0x7f4738074030 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473cd40700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738074030 0x7f473819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473cd40700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738074030 0x7f473819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57274/0 (socket says 172.21.15.192:57274) 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473cd40700 1 -- 172.21.15.192:0/4265292108 learned_addr learned my addr 172.21.15.192:0/4265292108 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.382+0000 7f473d541700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f473810dc00 0x7f47381a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f4737fff700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4738075600 0x7f473819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f473cd40700 1 -- 172.21.15.192:0/4265292108 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4738075600 msgr2=0x7f473819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f473cd40700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4738075600 0x7f473819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f473cd40700 1 -- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f473810dc00 msgr2=0x7f47381a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:11.385 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f473cd40700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f473810dc00 0x7f47381a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f473cd40700 1 -- 172.21.15.192:0/4265292108 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c0095f0 con 0x7f4738074030 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f4737fff700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4738075600 0x7f473819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f473cd40700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738074030 0x7f473819da70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f472c000c00 tx=0x7f472c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f4735ffb700 1 -- 172.21.15.192:0/4265292108 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c01d070 con 0x7f4738074030 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f4735ffb700 1 -- 172.21.15.192:0/4265292108 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f472c004080 con 0x7f4738074030 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47381a5b00 con 0x7f4738074030 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f4735ffb700 1 -- 172.21.15.192:0/4265292108 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c022c40 con 0x7f4738074030 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.383+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47381a5f70 con 0x7f4738074030 2023-12-17T12:08:11.386 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.384+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4738103e60 con 0x7f4738074030 2023-12-17T12:08:11.387 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.386+0000 7f4735ffb700 1 -- 172.21.15.192:0/4265292108 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f472c022470 con 0x7f4738074030 2023-12-17T12:08:11.388 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.387+0000 7f4735ffb700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47280a1500 0x7f47280a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:11.388 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.388+0000 7f4735ffb700 1 -- 172.21.15.192:0/4265292108 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f472c0cde40 con 0x7f4738074030 2023-12-17T12:08:11.389 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.388+0000 7f4737fff700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47280a1500 0x7f47280a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:11.389 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.389+0000 7f4737fff700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47280a1500 0x7f47280a39c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4724005fd0 tx=0x7f4724005f00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:11.392 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.392+0000 7f4735ffb700 1 -- 172.21.15.192:0/4265292108 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f472c097930 con 0x7f4738074030 2023-12-17T12:08:11.602 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.601+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f473802d4b0 con 0x7f47280a1500 2023-12-17T12:08:11.611 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.610+0000 7f4735ffb700 1 -- 172.21.15.192:0/4265292108 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f473802d4b0 con 0x7f47280a1500 2023-12-17T12:08:11.611 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2023-12-17T12:08:11.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47280a1500 msgr2=0x7f47280a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:11.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47280a1500 0x7f47280a39c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4724005fd0 tx=0x7f4724005f00 comp rx=0 tx=0).stop 2023-12-17T12:08:11.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738074030 msgr2=0x7f473819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:11.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738074030 0x7f473819da70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f472c000c00 tx=0x7f472c005f10 comp rx=0 tx=0).stop 2023-12-17T12:08:11.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 shutdown_connections 2023-12-17T12:08:11.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f47280a1500 0x7f47280a39c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4738074030 0x7f473819da70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f473810dc00 0x7f47381a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 --2- 172.21.15.192:0/4265292108 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4738075600 0x7f473819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:11.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 >> 172.21.15.192:0/4265292108 conn(0x7f47380fcbe0 msgr2=0x7f4738100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:11.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 shutdown_connections 2023-12-17T12:08:11.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-17T12:08:11.613+0000 7f473efa4700 1 -- 172.21.15.192:0/4265292108 wait complete. 2023-12-17T12:08:12.238 DEBUG:teuthology.orchestra.run.smithi027:mds.a> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.a.service 2023-12-17T12:08:12.244 DEBUG:teuthology.orchestra.run.smithi027:mds.d> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.d.service 2023-12-17T12:08:12.289 DEBUG:teuthology.orchestra.run.smithi027:mds.g> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.g.service 2023-12-17T12:08:12.329 INFO:journalctl@ceph.mds.a.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:08:12.334 DEBUG:teuthology.orchestra.run.smithi027:mds.j> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.j.service 2023-12-17T12:08:12.367 INFO:journalctl@ceph.mds.d.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:08:12.371 DEBUG:teuthology.orchestra.run.smithi141:mds.b> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.b.service 2023-12-17T12:08:12.377 DEBUG:teuthology.orchestra.run.smithi141:mds.e> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.e.service 2023-12-17T12:08:12.414 INFO:journalctl@ceph.mds.g.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:08:12.423 DEBUG:teuthology.orchestra.run.smithi141:mds.h> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.h.service 2023-12-17T12:08:12.445 INFO:journalctl@ceph.mds.j.smithi027.stdout:-- Logs begin at Sun 2023-12-17 11:42:56 UTC. -- 2023-12-17T12:08:12.448 INFO:journalctl@ceph.mds.b.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:08:12.452 DEBUG:teuthology.orchestra.run.smithi141:mds.k> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.k.service 2023-12-17T12:08:12.491 INFO:journalctl@ceph.mds.e.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:08:12.496 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.c.service 2023-12-17T12:08:12.501 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.f.service 2023-12-17T12:08:12.522 INFO:journalctl@ceph.mds.h.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:08:12.547 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.i.service 2023-12-17T12:08:12.563 INFO:journalctl@ceph.mds.k.smithi141.stdout:-- Logs begin at Sun 2023-12-17 11:40:37 UTC. -- 2023-12-17T12:08:12.575 INFO:journalctl@ceph.mds.c.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:08:12.581 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.l.service 2023-12-17T12:08:12.619 INFO:journalctl@ceph.mds.f.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:08:12.622 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-12-17T12:08:12.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:08:12.653 INFO:journalctl@ceph.mds.i.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:08:12.693 INFO:journalctl@ceph.mds.l.smithi192.stdout:-- Logs begin at Sun 2023-12-17 11:38:39 UTC. -- 2023-12-17T12:08:12.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:12.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:12.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-12-17T12:08:12.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:12.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:12.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:12.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:12.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:12.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:12.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:12.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:12 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:12.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 -- 172.21.15.27:0/671222896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 msgr2=0x7f73681095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 --2- 172.21.15.27:0/671222896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 0x7f73681095d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f736000b0a0 tx=0x7f736000b3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 -- 172.21.15.27:0/671222896 shutdown_connections 2023-12-17T12:08:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 --2- 172.21.15.27:0/671222896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 0x7f73681095d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 --2- 172.21.15.27:0/671222896 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7368101f00 0x7f73681042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 --2- 172.21.15.27:0/671222896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7368104830 0x7f7368106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 -- 172.21.15.27:0/671222896 >> 172.21.15.27:0/671222896 conn(0x7f73680fb8f0 msgr2=0x7f73680fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 -- 172.21.15.27:0/671222896 shutdown_connections 2023-12-17T12:08:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.841+0000 7f736d5ff700 1 -- 172.21.15.27:0/671222896 wait complete. 2023-12-17T12:08:12.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.842+0000 7f736d5ff700 1 Processor -- start 2023-12-17T12:08:12.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.842+0000 7f736d5ff700 1 -- start start 2023-12-17T12:08:12.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.842+0000 7f736d5ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7368101f00 0x7f73681995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.842+0000 7f736d5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7368104830 0x7f7368199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.842+0000 7f736d5ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 0x7f73681a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f736d5ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73681a1120 con 0x7f7368107160 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f736d5ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73681a1260 con 0x7f7368101f00 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f736d5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73681a13f0 con 0x7f7368104830 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 0x7f73681a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f735ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7368104830 0x7f7368199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 0x7f73681a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33154/0 (socket says 172.21.15.27:33154) 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 -- 172.21.15.27:0/2283057196 learned_addr learned my addr 172.21.15.27:0/2283057196 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f7366ffd700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7368101f00 0x7f73681995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 -- 172.21.15.27:0/2283057196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7368104830 msgr2=0x7f7368199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:12.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7368104830 0x7f7368199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 -- 172.21.15.27:0/2283057196 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7368101f00 msgr2=0x7f73681995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7368101f00 0x7f73681995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 -- 172.21.15.27:0/2283057196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360009d00 con 0x7f7368107160 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f735ffff700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7368104830 0x7f7368199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.843+0000 7f73677fe700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 0x7f73681a0b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f736000b780 tx=0x7f736000bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.844+0000 7f7364ff9700 1 -- 172.21.15.27:0/2283057196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000e040 con 0x7f7368107160 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.844+0000 7f7364ff9700 1 -- 172.21.15.27:0/2283057196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7360009480 con 0x7f7368107160 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.844+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73681a1670 con 0x7f7368107160 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.844+0000 7f7364ff9700 1 -- 172.21.15.27:0/2283057196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73600052c0 con 0x7f7368107160 2023-12-17T12:08:12.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.844+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73681a1a90 con 0x7f7368107160 2023-12-17T12:08:12.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.845+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7368108ce0 con 0x7f7368107160 2023-12-17T12:08:12.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.847+0000 7f7364ff9700 1 -- 172.21.15.27:0/2283057196 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7360005420 con 0x7f7368107160 2023-12-17T12:08:12.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.849+0000 7f7364ff9700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73540a1480 0x7f73540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:12.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.849+0000 7f7364ff9700 1 -- 172.21.15.27:0/2283057196 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73600cda20 con 0x7f7368107160 2023-12-17T12:08:12.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.849+0000 7f7366ffd700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73540a1480 0x7f73540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:12.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.850+0000 7f7366ffd700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73540a1480 0x7f73540a3940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f735800bbd0 tx=0x7f735800b5b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:12.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:12.854+0000 7f7364ff9700 1 -- 172.21.15.27:0/2283057196 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7360097510 con 0x7f7368107160 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:12.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:12 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:13.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:13.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:12 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.070+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7368068230 con 0x7f7368107160 2023-12-17T12:08:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.071+0000 7f7364ff9700 1 -- 172.21.15.27:0/2283057196 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18433 (secure 0 0 0) 0x7f736009ab30 con 0x7f7368107160 2023-12-17T12:08:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":78,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:08:03.382807+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73540a1480 msgr2=0x7f73540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73540a1480 0x7f73540a3940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f735800bbd0 tx=0x7f735800b5b0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 msgr2=0x7f73681a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 0x7f73681a0b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f736000b780 tx=0x7f736000bcc0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 shutdown_connections 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73540a1480 0x7f73540a3940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7368107160 0x7f73681a0b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7368101f00 0x7f73681995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 --2- 172.21.15.27:0/2283057196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7368104830 0x7f7368199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 >> 172.21.15.27:0/2283057196 conn(0x7f73680fb8f0 msgr2=0x7f73681016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:13.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 shutdown_connections 2023-12-17T12:08:13.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.074+0000 7f736d5ff700 1 -- 172.21.15.27:0/2283057196 wait complete. 2023-12-17T12:08:13.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-17T12:04:41.530587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-17T12:08:13.096 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 -- 172.21.15.27:0/2788270425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 msgr2=0x7f4e50074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 --2- 172.21.15.27:0/2788270425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 0x7f4e50074450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098f0 tx=0x7f4e3c009c00 comp rx=0 tx=0).stop 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 -- 172.21.15.27:0/2788270425 shutdown_connections 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 --2- 172.21.15.27:0/2788270425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e50109470 0x7f4e5010b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 --2- 172.21.15.27:0/2788270425 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e500755b0 0x7f4e50073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 --2- 172.21.15.27:0/2788270425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 0x7f4e50074450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 -- 172.21.15.27:0/2788270425 >> 172.21.15.27:0/2788270425 conn(0x7f4e500fcb10 msgr2=0x7f4e500fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 -- 172.21.15.27:0/2788270425 shutdown_connections 2023-12-17T12:08:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.306+0000 7f4e571ff700 1 -- 172.21.15.27:0/2788270425 wait complete. 2023-12-17T12:08:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.307+0000 7f4e571ff700 1 Processor -- start 2023-12-17T12:08:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.307+0000 7f4e571ff700 1 -- start start 2023-12-17T12:08:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.307+0000 7f4e571ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 0x7f4e50199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e571ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e500755b0 0x7f4e50199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e571ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e50109470 0x7f4e501a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e571ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501a11b0 con 0x7f4e500755b0 2023-12-17T12:08:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e571ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501a12f0 con 0x7f4e50109470 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e571ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501a1480 con 0x7f4e50073fd0 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e4ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e500755b0 0x7f4e50199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e54f9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 0x7f4e50199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e54f9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 0x7f4e50199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:43462/0 (socket says 172.21.15.27:43462) 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e54f9b700 1 -- 172.21.15.27:0/3874414090 learned_addr learned my addr 172.21.15.27:0/3874414090 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e5579c700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e50109470 0x7f4e501a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e54f9b700 1 -- 172.21.15.27:0/3874414090 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e50109470 msgr2=0x7f4e501a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e54f9b700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e50109470 0x7f4e501a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e54f9b700 1 -- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e500755b0 msgr2=0x7f4e50199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e54f9b700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e500755b0 0x7f4e50199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e54f9b700 1 -- 172.21.15.27:0/3874414090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c009520 con 0x7f4e50073fd0 2023-12-17T12:08:13.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.308+0000 7f4e4ffff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e500755b0 0x7f4e50199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:13.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.309+0000 7f4e54f9b700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 0x7f4e50199680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400e9a0 tx=0x7f4e4400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:13.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.309+0000 7f4e4dffb700 1 -- 172.21.15.27:0/3874414090 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44009790 con 0x7f4e50073fd0 2023-12-17T12:08:13.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.309+0000 7f4e4dffb700 1 -- 172.21.15.27:0/3874414090 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e44010430 con 0x7f4e50073fd0 2023-12-17T12:08:13.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.309+0000 7f4e4dffb700 1 -- 172.21.15.27:0/3874414090 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400cea0 con 0x7f4e50073fd0 2023-12-17T12:08:13.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.309+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e501a1760 con 0x7f4e50073fd0 2023-12-17T12:08:13.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.309+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e501a1c20 con 0x7f4e50073fd0 2023-12-17T12:08:13.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.310+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e50108280 con 0x7f4e50073fd0 2023-12-17T12:08:13.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.312+0000 7f4e4dffb700 1 -- 172.21.15.27:0/3874414090 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4e440105a0 con 0x7f4e50073fd0 2023-12-17T12:08:13.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.313+0000 7f4e4dffb700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e400a1500 0x7f4e400a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:13.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.313+0000 7f4e4dffb700 1 -- 172.21.15.27:0/3874414090 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4e44014070 con 0x7f4e50073fd0 2023-12-17T12:08:13.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.314+0000 7f4e4ffff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e400a1500 0x7f4e400a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:13.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.314+0000 7f4e4ffff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e400a1500 0x7f4e400a39c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00f9d0 tx=0x7f4e3c019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.317+0000 7f4e4dffb700 1 -- 172.21.15.27:0/3874414090 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4e44096480 con 0x7f4e50073fd0 2023-12-17T12:08:13.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.529+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e5010f140 con 0x7f4e50073fd0 2023-12-17T12:08:13.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.529+0000 7f4e4dffb700 1 -- 172.21.15.27:0/3874414090 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e44099aa0 con 0x7f4e50073fd0 2023-12-17T12:08:13.531 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-17T12:08:13.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e400a1500 msgr2=0x7f4e400a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e400a1500 0x7f4e400a39c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00f9d0 tx=0x7f4e3c019040 comp rx=0 tx=0).stop 2023-12-17T12:08:13.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 msgr2=0x7f4e50199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 0x7f4e50199680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400e9a0 tx=0x7f4e4400ecb0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 shutdown_connections 2023-12-17T12:08:13.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e400a1500 0x7f4e400a39c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e50073fd0 0x7f4e50199680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e50109470 0x7f4e501a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 --2- 172.21.15.27:0/3874414090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e500755b0 0x7f4e50199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 >> 172.21.15.27:0/3874414090 conn(0x7f4e500fcb10 msgr2=0x7f4e501053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 shutdown_connections 2023-12-17T12:08:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.532+0000 7f4e571ff700 1 -- 172.21.15.27:0/3874414090 wait complete. 2023-12-17T12:08:13.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:13.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:13 smithi192 ceph-mon[41899]: Deploying daemon mds.l on smithi192 2023-12-17T12:08:13.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:13 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2283057196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:13.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:13 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3874414090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 -- 172.21.15.27:0/288009189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 msgr2=0x7f852810b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 --2- 172.21.15.27:0/288009189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 0x7f852810b7f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f852000b0a0 tx=0x7f852000b3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 -- 172.21.15.27:0/288009189 shutdown_connections 2023-12-17T12:08:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 --2- 172.21.15.27:0/288009189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8528075610 0x7f8528073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 --2- 172.21.15.27:0/288009189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 0x7f852810b7f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 --2- 172.21.15.27:0/288009189 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8528074030 0x7f85280744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 -- 172.21.15.27:0/288009189 >> 172.21.15.27:0/288009189 conn(0x7f85280fca10 msgr2=0x7f85280fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:13.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 -- 172.21.15.27:0/288009189 shutdown_connections 2023-12-17T12:08:13.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.762+0000 7f852f9ff700 1 -- 172.21.15.27:0/288009189 wait complete. 2023-12-17T12:08:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.763+0000 7f852f9ff700 1 Processor -- start 2023-12-17T12:08:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.763+0000 7f852f9ff700 1 -- start start 2023-12-17T12:08:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.763+0000 7f852f9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8528074030 0x7f8528199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852f9ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8528075610 0x7f8528199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852f9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 0x7f85281a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852f9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85281a1150 con 0x7f8528109360 2023-12-17T12:08:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852f9ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85281a1290 con 0x7f8528075610 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852f9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85281a1420 con 0x7f8528074030 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 0x7f85281a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 0x7f85281a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33190/0 (socket says 172.21.15.27:33190) 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 -- 172.21.15.27:0/1536299897 learned_addr learned my addr 172.21.15.27:0/1536299897 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852cf9a700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8528075610 0x7f8528199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 -- 172.21.15.27:0/1536299897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8528074030 msgr2=0x7f8528199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8528074030 0x7f8528199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 -- 172.21.15.27:0/1536299897 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8528075610 msgr2=0x7f8528199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8528075610 0x7f8528199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 -- 172.21.15.27:0/1536299897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8520009d00 con 0x7f8528109360 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852cf9a700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8528075610 0x7f8528199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.764+0000 7f852df9c700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 0x7f85281a0b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f852000b780 tx=0x7f852000bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.765+0000 7f851e7fc700 1 -- 172.21.15.27:0/1536299897 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000e040 con 0x7f8528109360 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.765+0000 7f851e7fc700 1 -- 172.21.15.27:0/1536299897 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f852000be00 con 0x7f8528109360 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.765+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85281a16a0 con 0x7f8528109360 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.765+0000 7f851e7fc700 1 -- 172.21.15.27:0/1536299897 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200052a0 con 0x7f8528109360 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.765+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85281a1b20 con 0x7f8528109360 2023-12-17T12:08:13.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.766+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8528108120 con 0x7f8528109360 2023-12-17T12:08:13.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.768+0000 7f851e7fc700 1 -- 172.21.15.27:0/1536299897 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f852001d9a0 con 0x7f8528109360 2023-12-17T12:08:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.769+0000 7f851e7fc700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f85140a1500 0x7f85140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.769+0000 7f851e7fc700 1 -- 172.21.15.27:0/1536299897 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f85200cda70 con 0x7f8528109360 2023-12-17T12:08:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.770+0000 7f852d79b700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f85140a1500 0x7f85140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.770+0000 7f852d79b700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f85140a1500 0x7f85140a39c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8524005950 tx=0x7f85240058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:13.773+0000 7f851e7fc700 1 -- 172.21.15.27:0/1536299897 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8520097560 con 0x7f8528109360 2023-12-17T12:08:13.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:13 smithi027 ceph-mon[36871]: Deploying daemon mds.l on smithi192 2023-12-17T12:08:13.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:13 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2283057196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:13.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:13 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3874414090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:14.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:13 smithi141 ceph-mon[41225]: Deploying daemon mds.l on smithi192 2023-12-17T12:08:14.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:13 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2283057196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:14.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:13 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3874414090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.032+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8528068230 con 0x7f8528109360 2023-12-17T12:08:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.032+0000 7f851e7fc700 1 -- 172.21.15.27:0/1536299897 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f852009ab80 con 0x7f8528109360 2023-12-17T12:08:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:14.034 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-12-17T12:08:14.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f85140a1500 msgr2=0x7f85140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:14.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f85140a1500 0x7f85140a39c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8524005950 tx=0x7f85240058e0 comp rx=0 tx=0).stop 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 msgr2=0x7f85281a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 0x7f85281a0b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f852000b780 tx=0x7f852000bb60 comp rx=0 tx=0).stop 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 shutdown_connections 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f85140a1500 0x7f85140a39c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8528074030 0x7f8528199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8528109360 0x7f85281a0b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 --2- 172.21.15.27:0/1536299897 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8528075610 0x7f8528199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 >> 172.21.15.27:0/1536299897 conn(0x7f85280fca10 msgr2=0x7f8528105260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 shutdown_connections 2023-12-17T12:08:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:14.035+0000 7f852f9ff700 1 -- 172.21.15.27:0/1536299897 wait complete. 2023-12-17T12:08:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 1 2023-12-17T12:08:14.300 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:08:14 smithi192 systemd[1]: Starting Ceph mds.l for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:14.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:14 smithi192 ceph-mon[41899]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:14.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:14 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1536299897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:14.872 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:08:14 smithi192 bash[77928]: 977b7e2de95ad067120d1e254800e3ddb217462eed27bd82ce8b4ec61af7ca32 2023-12-17T12:08:14.872 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:08:14 smithi192 systemd[1]: Started Ceph mds.l for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:14.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:14 smithi027 ceph-mon[36871]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:14.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:14 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1536299897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:15.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:14 smithi141 ceph-mon[41225]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:15.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:14 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1536299897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:15.310 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:08:14 smithi192 ceph-mds[77966]: -- 172.21.15.192:0/2595070780 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c706775520 con 0x55c70674d000 2023-12-17T12:08:15.310 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:08:14 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-l[77943]: starting mds.l at 2023-12-17T12:08:16.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:16.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.192:6832/2710858125,v1:172.21.15.192:6833/2710858125] up:boot 2023-12-17T12:08:16.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: fsmap 1 up:standby 2023-12-17T12:08:16.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-17T12:08:16.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:16.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:16.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:16.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:15 smithi141 ceph-mon[41225]: Deploying daemon mds.k on smithi141 2023-12-17T12:08:16.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.192:6832/2710858125,v1:172.21.15.192:6833/2710858125] up:boot 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: fsmap 1 up:standby 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:16.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:15 smithi027 ceph-mon[36871]: Deploying daemon mds.k on smithi141 2023-12-17T12:08:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 -- 172.21.15.27:0/1497112356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a41014a0 msgr2=0x7f38a41018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 --2- 172.21.15.27:0/1497112356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a41014a0 0x7f38a41018c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38940099c0 tx=0x7f3894009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 -- 172.21.15.27:0/1497112356 shutdown_connections 2023-12-17T12:08:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 --2- 172.21.15.27:0/1497112356 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f38a4103030 0x7f38a41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 --2- 172.21.15.27:0/1497112356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a41014a0 0x7f38a41018c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 --2- 172.21.15.27:0/1497112356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38a4102690 0x7f38a4102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 -- 172.21.15.27:0/1497112356 >> 172.21.15.27:0/1497112356 conn(0x7f38a40fca10 msgr2=0x7f38a40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 -- 172.21.15.27:0/1497112356 shutdown_connections 2023-12-17T12:08:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.275+0000 7f38a89ff700 1 -- 172.21.15.27:0/1497112356 wait complete. 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.276+0000 7f38a89ff700 1 Processor -- start 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.276+0000 7f38a89ff700 1 -- start start 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.276+0000 7f38a89ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38a41014a0 0x7f38a4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.276+0000 7f38a89ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f38a4102690 0x7f38a4199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.276+0000 7f38a89ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a4103030 0x7f38a41a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.276+0000 7f38a89ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41a10b0 con 0x7f38a4103030 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.276+0000 7f38a89ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41a11f0 con 0x7f38a4102690 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.276+0000 7f38a89ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41a1380 con 0x7f38a41014a0 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a4103030 0x7f38a41a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f389bfff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f38a4102690 0x7f38a4199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a4103030 0x7f38a41a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36184/0 (socket says 172.21.15.27:36184) 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 -- 172.21.15.27:0/2876594465 learned_addr learned my addr 172.21.15.27:0/2876594465 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a259c700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38a41014a0 0x7f38a4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 -- 172.21.15.27:0/2876594465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38a41014a0 msgr2=0x7f38a4199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38a41014a0 0x7f38a4199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 -- 172.21.15.27:0/2876594465 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f38a4102690 msgr2=0x7f38a4199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f38a4102690 0x7f38a4199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 -- 172.21.15.27:0/2876594465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940095f0 con 0x7f38a4103030 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a259c700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38a41014a0 0x7f38a4199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.277+0000 7f38a2d9d700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a4103030 0x7f38a41a0ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f389c007ee0 tx=0x7f389c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.278+0000 7f389b7fe700 1 -- 172.21.15.27:0/2876594465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c017070 con 0x7f38a4103030 2023-12-17T12:08:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.278+0000 7f389b7fe700 1 -- 172.21.15.27:0/2876594465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f389c00f040 con 0x7f38a4103030 2023-12-17T12:08:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.278+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a41a1660 con 0x7f38a4103030 2023-12-17T12:08:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.278+0000 7f389b7fe700 1 -- 172.21.15.27:0/2876594465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c005710 con 0x7f38a4103030 2023-12-17T12:08:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.278+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a41a1b70 con 0x7f38a4103030 2023-12-17T12:08:16.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.279+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4068230 con 0x7f38a4103030 2023-12-17T12:08:16.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.282+0000 7f389b7fe700 1 -- 172.21.15.27:0/2876594465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f389c005870 con 0x7f38a4103030 2023-12-17T12:08:16.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.284+0000 7f389b7fe700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f38900a1500 0x7f38900a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:16.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.284+0000 7f389b7fe700 1 -- 172.21.15.27:0/2876594465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f389c0cc5f0 con 0x7f38a4103030 2023-12-17T12:08:16.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.284+0000 7f38a259c700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f38900a1500 0x7f38900a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:16.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.285+0000 7f38a259c700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f38900a1500 0x7f38900a39c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3894000c00 tx=0x7f3894005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.286+0000 7f389b7fe700 1 -- 172.21.15.27:0/2876594465 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f389c0960f0 con 0x7f38a4103030 2023-12-17T12:08:16.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:16.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.192:6832/2710858125,v1:172.21.15.192:6833/2710858125] up:boot 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: fsmap 1 up:standby 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:16.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:15 smithi192 ceph-mon[41899]: Deploying daemon mds.k on smithi141 2023-12-17T12:08:16.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.543+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38a410cbc0 con 0x7f38a4103030 2023-12-17T12:08:16.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.544+0000 7f389b7fe700 1 -- 172.21.15.27:0/2876594465 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f389c099710 con 0x7f38a4103030 2023-12-17T12:08:16.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:16.546 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-17T12:08:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f38900a1500 msgr2=0x7f38900a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f38900a1500 0x7f38900a39c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3894000c00 tx=0x7f3894005fb0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a4103030 msgr2=0x7f38a41a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a4103030 0x7f38a41a0ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f389c007ee0 tx=0x7f389c00d3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 shutdown_connections 2023-12-17T12:08:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f38900a1500 0x7f38900a39c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38a41014a0 0x7f38a4199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a4103030 0x7f38a41a0ae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 --2- 172.21.15.27:0/2876594465 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f38a4102690 0x7f38a4199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:16.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 >> 172.21.15.27:0/2876594465 conn(0x7f38a40fca10 msgr2=0x7f38a4107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:16.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 shutdown_connections 2023-12-17T12:08:16.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:16.546+0000 7f38a89ff700 1 -- 172.21.15.27:0/2876594465 wait complete. 2023-12-17T12:08:16.549 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 2 2023-12-17T12:08:16.922 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:16 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2876594465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:17.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:16 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2876594465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:17.264 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:08:16 smithi141 systemd[1]: Starting Ceph mds.k for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:17.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:16 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2876594465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:17.764 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:08:17 smithi141 bash[76400]: a59bf74a49b047c7673fc6abefa6c2595deebb0f09e3d9c5da60892cb897a2e7 2023-12-17T12:08:17.764 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:08:17 smithi141 systemd[1]: Started Ceph mds.k for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:17.764 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:08:17 smithi141 ceph-mds[76438]: -- 172.21.15.141:0/1858539692 <== mon.2 v2:172.21.15.141:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dbb8405520 con 0x55dbb83dd000 2023-12-17T12:08:17.765 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:08:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-k[76415]: starting mds.k at 2023-12-17T12:08:18.569 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: Deploying daemon mds.j on smithi027 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6832/2577446255,v1:172.21.15.141:6833/2577446255] up:boot 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: fsmap 2 up:standby 2023-12-17T12:08:18.636 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:18 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.783+0000 7fad40bff700 1 -- 172.21.15.27:0/1712610667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c101f60 msgr2=0x7fad3c104350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.783+0000 7fad40bff700 1 --2- 172.21.15.27:0/1712610667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c101f60 0x7fad3c104350 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009990 tx=0x7fad2c009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.783+0000 7fad40bff700 1 -- 172.21.15.27:0/1712610667 shutdown_connections 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.783+0000 7fad40bff700 1 --2- 172.21.15.27:0/1712610667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c1071c0 0x7fad3c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.783+0000 7fad40bff700 1 --2- 172.21.15.27:0/1712610667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c101f60 0x7fad3c104350 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.783+0000 7fad40bff700 1 --2- 172.21.15.27:0/1712610667 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fad3c104890 0x7fad3c106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.783+0000 7fad40bff700 1 -- 172.21.15.27:0/1712610667 >> 172.21.15.27:0/1712610667 conn(0x7fad3c0fb8f0 msgr2=0x7fad3c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.783+0000 7fad40bff700 1 -- 172.21.15.27:0/1712610667 shutdown_connections 2023-12-17T12:08:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.784+0000 7fad40bff700 1 -- 172.21.15.27:0/1712610667 wait complete. 2023-12-17T12:08:18.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.785+0000 7fad40bff700 1 Processor -- start 2023-12-17T12:08:18.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.785+0000 7fad40bff700 1 -- start start 2023-12-17T12:08:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad40bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c101f60 0x7fad3c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad40bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c104890 0x7fad3c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad40bff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fad3c1071c0 0x7fad3c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad40bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19ee60 con 0x7fad3c104890 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad40bff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19efa0 con 0x7fad3c1071c0 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad40bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19f130 con 0x7fad3c101f60 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad39d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c104890 0x7fad3c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad39d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c104890 0x7fad3c197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36208/0 (socket says 172.21.15.27:36208) 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad39d9b700 1 -- 172.21.15.27:0/1119971169 learned_addr learned my addr 172.21.15.27:0/1119971169 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad3a59c700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c101f60 0x7fad3c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad3ad9d700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fad3c1071c0 0x7fad3c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad39d9b700 1 -- 172.21.15.27:0/1119971169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c101f60 msgr2=0x7fad3c197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad39d9b700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c101f60 0x7fad3c197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad39d9b700 1 -- 172.21.15.27:0/1119971169 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fad3c1071c0 msgr2=0x7fad3c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad39d9b700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fad3c1071c0 0x7fad3c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad39d9b700 1 -- 172.21.15.27:0/1119971169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c0095f0 con 0x7fad3c104890 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.786+0000 7fad3a59c700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c101f60 0x7fad3c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:08:18.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.787+0000 7fad39d9b700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c104890 0x7fad3c197870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fad3000d950 tx=0x7fad3000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.787+0000 7fad2b7fe700 1 -- 172.21.15.27:0/1119971169 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30009b40 con 0x7fad3c104890 2023-12-17T12:08:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.787+0000 7fad2b7fe700 1 -- 172.21.15.27:0/1119971169 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad30009ca0 con 0x7fad3c104890 2023-12-17T12:08:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.787+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c19f410 con 0x7fad3c104890 2023-12-17T12:08:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.787+0000 7fad2b7fe700 1 -- 172.21.15.27:0/1119971169 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3000f620 con 0x7fad3c104890 2023-12-17T12:08:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.787+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c19f970 con 0x7fad3c104890 2023-12-17T12:08:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.788+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c0fd470 con 0x7fad3c104890 2023-12-17T12:08:18.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.790+0000 7fad2b7fe700 1 -- 172.21.15.27:0/1119971169 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fad30010460 con 0x7fad3c104890 2023-12-17T12:08:18.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.791+0000 7fad2b7fe700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fad240a1500 0x7fad240a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:18.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.792+0000 7fad2b7fe700 1 -- 172.21.15.27:0/1119971169 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fad300ccc20 con 0x7fad3c104890 2023-12-17T12:08:18.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.792+0000 7fad3a59c700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fad240a1500 0x7fad240a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:18.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.792+0000 7fad3a59c700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fad240a1500 0x7fad240a39c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad2c003780 tx=0x7fad2c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:18.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:18.795+0000 7fad2b7fe700 1 -- 172.21.15.27:0/1119971169 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fad30096680 con 0x7fad3c104890 2023-12-17T12:08:19.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:19.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:19.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:19.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:19.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:19.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:19.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:19.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: Deploying daemon mds.j on smithi027 2023-12-17T12:08:19.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6832/2577446255,v1:172.21.15.141:6833/2577446255] up:boot 2023-12-17T12:08:19.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: fsmap 2 up:standby 2023-12-17T12:08:19.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:18 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-17T12:08:19.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: Deploying daemon mds.j on smithi027 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6832/2577446255,v1:172.21.15.141:6833/2577446255] up:boot 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: fsmap 2 up:standby 2023-12-17T12:08:19.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:18 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-17T12:08:19.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.084+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad3c068230 con 0x7fad3c104890 2023-12-17T12:08:19.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.084+0000 7fad2b7fe700 1 -- 172.21.15.27:0/1119971169 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fad30099ca0 con 0x7fad3c104890 2023-12-17T12:08:19.086 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:19.086 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fad240a1500 msgr2=0x7fad240a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fad240a1500 0x7fad240a39c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad2c003780 tx=0x7fad2c005fb0 comp rx=0 tx=0).stop 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c104890 msgr2=0x7fad3c197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c104890 0x7fad3c197870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fad3000d950 tx=0x7fad3000dd10 comp rx=0 tx=0).stop 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 shutdown_connections 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fad240a1500 0x7fad240a39c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c101f60 0x7fad3c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fad3c1071c0 0x7fad3c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 --2- 172.21.15.27:0/1119971169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad3c104890 0x7fad3c197870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 >> 172.21.15.27:0/1119971169 conn(0x7fad3c0fb8f0 msgr2=0x7fad3c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 shutdown_connections 2023-12-17T12:08:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:19.087+0000 7fad40bff700 1 -- 172.21.15.27:0/1119971169 wait complete. 2023-12-17T12:08:19.090 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2023-12-17T12:08:19.871 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:08:19 smithi027 systemd[1]: Starting Ceph mds.j for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:20.122 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:08:20 smithi027 bash[87326]: 282d98a64b3331a0c95c20e88054fd423a02ed43cd48194070f585efc6bf8acc 2023-12-17T12:08:20.122 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:08:20 smithi027 systemd[1]: Started Ceph mds.j for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:20.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:19 smithi027 ceph-mon[36871]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:20.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:19 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1119971169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:20.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:19 smithi141 ceph-mon[41225]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:20.264 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:19 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1119971169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:20.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:19 smithi192 ceph-mon[41899]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:20.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:19 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1119971169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:20.412 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:08:20 smithi027 ceph-mds[87364]: -- 172.21.15.27:0/3652494103 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5633fa12b520 con 0x5633fa102c00 2023-12-17T12:08:20.412 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:08:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-j[87341]: starting mds.j at 2023-12-17T12:08:21.110 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:21.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 -- 172.21.15.27:0/1978708809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230103030 msgr2=0x7f12301098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:21.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 --2- 172.21.15.27:0/1978708809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230103030 0x7f12301098e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f122800b0d0 tx=0x7f122800b3e0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 -- 172.21.15.27:0/1978708809 shutdown_connections 2023-12-17T12:08:21.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 --2- 172.21.15.27:0/1978708809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230103030 0x7f12301098e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 --2- 172.21.15.27:0/1978708809 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12301014a0 0x7f12301018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 --2- 172.21.15.27:0/1978708809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1230102690 0x7f1230102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 -- 172.21.15.27:0/1978708809 >> 172.21.15.27:0/1978708809 conn(0x7f12300fca10 msgr2=0x7f12300fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:21.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 -- 172.21.15.27:0/1978708809 shutdown_connections 2023-12-17T12:08:21.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.330+0000 7f12381ff700 1 -- 172.21.15.27:0/1978708809 wait complete. 2023-12-17T12:08:21.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.331+0000 7f12381ff700 1 Processor -- start 2023-12-17T12:08:21.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f12381ff700 1 -- start start 2023-12-17T12:08:21.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f12381ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12301014a0 0x7f1230075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:21.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f12381ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230102690 0x7f12300739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:21.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f12381ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1230103030 0x7f1230073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f12381ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12300746a0 con 0x7f1230102690 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f12381ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1230074810 con 0x7f12301014a0 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f12381ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12301a1480 con 0x7f1230103030 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f123579a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230102690 0x7f12300739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f123579a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230102690 0x7f12300739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36250/0 (socket says 172.21.15.27:36250) 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f123579a700 1 -- 172.21.15.27:0/1810193249 learned_addr learned my addr 172.21.15.27:0/1810193249 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f1235f9b700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12301014a0 0x7f1230075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f123679c700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1230103030 0x7f1230073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f123579a700 1 -- 172.21.15.27:0/1810193249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1230103030 msgr2=0x7f1230073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:21.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.332+0000 7f123579a700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1230103030 0x7f1230073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f123579a700 1 -- 172.21.15.27:0/1810193249 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12301014a0 msgr2=0x7f1230075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f123579a700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12301014a0 0x7f1230075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f123579a700 1 -- 172.21.15.27:0/1810193249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228009d00 con 0x7f1230102690 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f1235f9b700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12301014a0 0x7f1230075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f123579a700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230102690 0x7f12300739c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f122000e9a0 tx=0x7f122000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f1226ffd700 1 -- 172.21.15.27:0/1810193249 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220009790 con 0x7f1230102690 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f1226ffd700 1 -- 172.21.15.27:0/1810193249 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1220010430 con 0x7f1230102690 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f1226ffd700 1 -- 172.21.15.27:0/1810193249 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220018680 con 0x7f1230102690 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12301a1710 con 0x7f1230102690 2023-12-17T12:08:21.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.333+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12301a1c20 con 0x7f1230102690 2023-12-17T12:08:21.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.334+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1230108970 con 0x7f1230102690 2023-12-17T12:08:21.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.336+0000 7f1226ffd700 1 -- 172.21.15.27:0/1810193249 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f12200105a0 con 0x7f1230102690 2023-12-17T12:08:21.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.338+0000 7f1226ffd700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f121c0a1500 0x7f121c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:21.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.338+0000 7f1226ffd700 1 -- 172.21.15.27:0/1810193249 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1220014070 con 0x7f1230102690 2023-12-17T12:08:21.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.338+0000 7f1235f9b700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f121c0a1500 0x7f121c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:21.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.339+0000 7f1235f9b700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f121c0a1500 0x7f121c0a39c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f12301024f0 tx=0x7f122c00b670 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:21.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.342+0000 7f1226ffd700 1 -- 172.21.15.27:0/1810193249 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f12200522f0 con 0x7f1230102690 2023-12-17T12:08:21.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:21.465 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:21.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:21.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: Deploying daemon mds.d on smithi027 2023-12-17T12:08:21.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.27:6834/2531226190,v1:172.21.15.27:6835/2531226190] up:boot 2023-12-17T12:08:21.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: fsmap 3 up:standby 2023-12-17T12:08:21.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:21 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-17T12:08:21.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:21.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:21.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:21.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: Deploying daemon mds.d on smithi027 2023-12-17T12:08:21.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.27:6834/2531226190,v1:172.21.15.27:6835/2531226190] up:boot 2023-12-17T12:08:21.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: fsmap 3 up:standby 2023-12-17T12:08:21.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:21 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-17T12:08:21.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: Deploying daemon mds.d on smithi027 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.27:6834/2531226190,v1:172.21.15.27:6835/2531226190] up:boot 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: fsmap 3 up:standby 2023-12-17T12:08:21.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:21 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-17T12:08:21.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.627+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1230068230 con 0x7f1230102690 2023-12-17T12:08:21.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.629+0000 7f1226ffd700 1 -- 172.21.15.27:0/1810193249 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f1220096540 con 0x7f1230102690 2023-12-17T12:08:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:21.631 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.633+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f121c0a1500 msgr2=0x7f121c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.633+0000 7f12381ff700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f121c0a1500 0x7f121c0a39c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f12301024f0 tx=0x7f122c00b670 comp rx=0 tx=0).stop 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.633+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230102690 msgr2=0x7f12300739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.633+0000 7f12381ff700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230102690 0x7f12300739c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f122000e9a0 tx=0x7f122000ed60 comp rx=0 tx=0).stop 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.634+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 shutdown_connections 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.634+0000 7f12381ff700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f121c0a1500 0x7f121c0a39c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.634+0000 7f12381ff700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1230103030 0x7f1230073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.634+0000 7f12381ff700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12301014a0 0x7f1230075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.634+0000 7f12381ff700 1 --2- 172.21.15.27:0/1810193249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1230102690 0x7f12300739c0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f122000e9a0 tx=0x7f122000ed60 comp rx=0 tx=0).stop 2023-12-17T12:08:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.634+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 >> 172.21.15.27:0/1810193249 conn(0x7f12300fca10 msgr2=0x7f1230105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.634+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 shutdown_connections 2023-12-17T12:08:21.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:21.634+0000 7f12381ff700 1 -- 172.21.15.27:0/1810193249 wait complete. 2023-12-17T12:08:21.637 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2023-12-17T12:08:22.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:22 smithi141 ceph-mon[41225]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:22.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:22 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1810193249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:22.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:22 smithi192 ceph-mon[41899]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:22.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:22 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1810193249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:22.635 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:22 smithi027 ceph-mon[36871]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:22.635 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:22 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1810193249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:22.636 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:08:22 smithi027 systemd[1]: Starting Ceph mds.d for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:23.162 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:08:22 smithi027 bash[87711]: 7db036f35d197715d293ea5e4e7cb0195077151336eb686cad0422fe96ce381b 2023-12-17T12:08:23.162 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:08:22 smithi027 systemd[1]: Started Ceph mds.d for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:23.162 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:08:22 smithi027 ceph-mds[87749]: -- 172.21.15.27:0/3075038760 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cf83343520 con 0x55cf8331b000 2023-12-17T12:08:23.162 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:08:22 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-d[87726]: starting mds.d at 2023-12-17T12:08:23.657 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:23.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/2730114313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08074030 msgr2=0x7f4d080744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:23.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/2730114313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08074030 0x7f4d080744b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009990 tx=0x7f4cfc009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:23.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/2730114313 shutdown_connections 2023-12-17T12:08:23.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/2730114313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d08075610 0x7f4d08073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:23.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/2730114313 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4d08109360 0x7f4d0810b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:23.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/2730114313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08074030 0x7f4d080744b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:23.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/2730114313 >> 172.21.15.27:0/2730114313 conn(0x7f4d080fca10 msgr2=0x7f4d080fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:23.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/2730114313 shutdown_connections 2023-12-17T12:08:23.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.881+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/2730114313 wait complete. 2023-12-17T12:08:23.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.882+0000 7f4d0e5ff700 1 Processor -- start 2023-12-17T12:08:23.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.882+0000 7f4d0e5ff700 1 -- start start 2023-12-17T12:08:23.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d0e5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d08074030 0x7f4d081994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d0e5ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08075610 0x7f4d08199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d0e5ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4d08109360 0x7f4d081a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d0e5ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d081a1000 con 0x7f4d08075610 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d0e5ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d081a1140 con 0x7f4d08109360 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d0e5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d081a12d0 con 0x7f4d08074030 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d077fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08075610 0x7f4d08199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d077fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08075610 0x7f4d08199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36294/0 (socket says 172.21.15.27:36294) 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d077fe700 1 -- 172.21.15.27:0/3800094645 learned_addr learned my addr 172.21.15.27:0/3800094645 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d0cb9c700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4d08109360 0x7f4d081a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d07fff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d08074030 0x7f4d081994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d077fe700 1 -- 172.21.15.27:0/3800094645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d08074030 msgr2=0x7f4d081994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d077fe700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d08074030 0x7f4d081994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d077fe700 1 -- 172.21.15.27:0/3800094645 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4d08109360 msgr2=0x7f4d081a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d077fe700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4d08109360 0x7f4d081a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:23.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.883+0000 7f4d077fe700 1 -- 172.21.15.27:0/3800094645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0095f0 con 0x7f4d08075610 2023-12-17T12:08:23.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.884+0000 7f4d077fe700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08075610 0x7f4d08199a10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005910 tx=0x7f4cfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.884+0000 7f4d057fa700 1 -- 172.21.15.27:0/3800094645 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc01d070 con 0x7f4d08075610 2023-12-17T12:08:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.884+0000 7f4d057fa700 1 -- 172.21.15.27:0/3800094645 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4cfc00bdb0 con 0x7f4d08075610 2023-12-17T12:08:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.884+0000 7f4d057fa700 1 -- 172.21.15.27:0/3800094645 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc022d40 con 0x7f4d08075610 2023-12-17T12:08:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.884+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d081a1550 con 0x7f4d08075610 2023-12-17T12:08:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.884+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d081a1970 con 0x7f4d08075610 2023-12-17T12:08:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.885+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d08108120 con 0x7f4d08075610 2023-12-17T12:08:23.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.887+0000 7f4d057fa700 1 -- 172.21.15.27:0/3800094645 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4cfc022470 con 0x7f4d08075610 2023-12-17T12:08:23.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.889+0000 7f4d057fa700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4cf00a1500 0x7f4cf00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:23.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.889+0000 7f4d057fa700 1 -- 172.21.15.27:0/3800094645 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4cfc0ce580 con 0x7f4d08075610 2023-12-17T12:08:23.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.889+0000 7f4d07fff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4cf00a1500 0x7f4cf00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:23.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.890+0000 7f4d07fff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4cf00a1500 0x7f4cf00a39c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800ac50 tx=0x7f4cf800a380 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:23.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:23.891+0000 7f4d057fa700 1 -- 172.21.15.27:0/3800094645 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4cfc098070 con 0x7f4d08075610 2023-12-17T12:08:23.923 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: Deploying daemon mds.e on smithi141 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.27:6836/2608720361,v1:172.21.15.27:6837/2608720361] up:boot 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: fsmap 4 up:standby 2023-12-17T12:08:23.924 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-17T12:08:24.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.149+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d08068230 con 0x7f4d08075610 2023-12-17T12:08:24.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.150+0000 7f4d057fa700 1 -- 172.21.15.27:0/3800094645 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f4cfc09b690 con 0x7f4d08075610 2023-12-17T12:08:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:24.150 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2023-12-17T12:08:24.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4cf00a1500 msgr2=0x7f4cf00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:24.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4cf00a1500 0x7f4cf00a39c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800ac50 tx=0x7f4cf800a380 comp rx=0 tx=0).stop 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08075610 msgr2=0x7f4d08199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08075610 0x7f4d08199a10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005910 tx=0x7f4cfc004730 comp rx=0 tx=0).stop 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 shutdown_connections 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4cf00a1500 0x7f4cf00a39c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d08074030 0x7f4d081994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4d08109360 0x7f4d081a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 --2- 172.21.15.27:0/3800094645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d08075610 0x7f4d08199a10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 >> 172.21.15.27:0/3800094645 conn(0x7f4d080fca10 msgr2=0x7f4d08105260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 shutdown_connections 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:24.152+0000 7f4d0e5ff700 1 -- 172.21.15.27:0/3800094645 wait complete. 2023-12-17T12:08:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 5 2023-12-17T12:08:24.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: Deploying daemon mds.e on smithi141 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.27:6836/2608720361,v1:172.21.15.27:6837/2608720361] up:boot 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: fsmap 4 up:standby 2023-12-17T12:08:24.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: Deploying daemon mds.e on smithi141 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.27:6836/2608720361,v1:172.21.15.27:6837/2608720361] up:boot 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: fsmap 4 up:standby 2023-12-17T12:08:24.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-17T12:08:25.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:24 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3800094645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:25.015 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:08:24 smithi141 systemd[1]: Starting Ceph mds.e for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:25.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:24 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3800094645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:25.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:24 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3800094645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:25.764 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:08:25 smithi141 ceph-mds[76778]: -- 172.21.15.141:0/1373672424 <== mon.2 v2:172.21.15.141:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e5b9c7520 con 0x558e5b99f000 2023-12-17T12:08:25.764 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:08:25 smithi141 bash[76740]: 48eacc27a9f1db03af1d226aad191b9e966731c48f6658b5fc113a2ab238b680 2023-12-17T12:08:25.764 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:08:25 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-e[76755]: starting mds.e at 2023-12-17T12:08:25.764 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:08:25 smithi141 systemd[1]: Started Ceph mds.e for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:26.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 -- 172.21.15.27:0/4012771450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8103030 msgr2=0x7fa7f81098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 --2- 172.21.15.27:0/4012771450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8103030 0x7fa7f81098e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000b0d0 tx=0x7fa7f000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 -- 172.21.15.27:0/4012771450 shutdown_connections 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 --2- 172.21.15.27:0/4012771450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8103030 0x7fa7f81098e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 --2- 172.21.15.27:0/4012771450 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa7f81014a0 0x7fa7f81018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 --2- 172.21.15.27:0/4012771450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7f8102690 0x7fa7f8102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 -- 172.21.15.27:0/4012771450 >> 172.21.15.27:0/4012771450 conn(0x7fa7f80fca10 msgr2=0x7fa7f80fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 -- 172.21.15.27:0/4012771450 shutdown_connections 2023-12-17T12:08:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.390+0000 7fa7fddff700 1 -- 172.21.15.27:0/4012771450 wait complete. 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.391+0000 7fa7fddff700 1 Processor -- start 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.391+0000 7fa7fddff700 1 -- start start 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7fddff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7f81014a0 0x7fa7f81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7fddff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8102690 0x7fa7f8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7fddff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa7f8103030 0x7fa7f819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7fddff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f819ef00 con 0x7fa7f8102690 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7fddff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f819f040 con 0x7fa7f8103030 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7fddff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f819f1d0 con 0x7fa7f81014a0 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8102690 0x7fa7f8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7f81014a0 0x7fa7f81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:26.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7f81014a0 0x7fa7f81973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:43452/0 (socket says 172.21.15.27:43452) 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f77fe700 1 -- 172.21.15.27:0/3547332855 learned_addr learned my addr 172.21.15.27:0/3547332855 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f7fff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa7f8103030 0x7fa7f819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f77fe700 1 -- 172.21.15.27:0/3547332855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa7f8103030 msgr2=0x7fa7f819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f77fe700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa7f8103030 0x7fa7f819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f77fe700 1 -- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8102690 msgr2=0x7fa7f8197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f77fe700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8102690 0x7fa7f8197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f77fe700 1 -- 172.21.15.27:0/3547332855 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f0009d00 con 0x7fa7f81014a0 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7effff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8102690 0x7fa7f8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.392+0000 7fa7f7fff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa7f8103030 0x7fa7f819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.393+0000 7fa7f77fe700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7f81014a0 0x7fa7f81973d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800e970 tx=0x7fa7e800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.393+0000 7fa7f57fa700 1 -- 172.21.15.27:0/3547332855 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8009790 con 0x7fa7f81014a0 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.393+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f819f4b0 con 0x7fa7f81014a0 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.393+0000 7fa7f57fa700 1 -- 172.21.15.27:0/3547332855 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7e800eec0 con 0x7fa7f81014a0 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.393+0000 7fa7f57fa700 1 -- 172.21.15.27:0/3547332855 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8003c10 con 0x7fa7f81014a0 2023-12-17T12:08:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.393+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f819f9c0 con 0x7fa7f81014a0 2023-12-17T12:08:26.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.394+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f81088c0 con 0x7fa7f81014a0 2023-12-17T12:08:26.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.396+0000 7fa7f57fa700 1 -- 172.21.15.27:0/3547332855 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa7e8003db0 con 0x7fa7f81014a0 2023-12-17T12:08:26.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.397+0000 7fa7f57fa700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa7e40a1480 0x7fa7e40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:26.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.397+0000 7fa7f57fa700 1 -- 172.21.15.27:0/3547332855 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7e8014070 con 0x7fa7f81014a0 2023-12-17T12:08:26.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.398+0000 7fa7effff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa7e40a1480 0x7fa7e40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:26.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.398+0000 7fa7effff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa7e40a1480 0x7fa7e40a3940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0011490 tx=0x7fa7e0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:26.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.402+0000 7fa7f57fa700 1 -- 172.21.15.27:0/3547332855 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa7e80965c0 con 0x7fa7f81014a0 2023-12-17T12:08:26.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.661+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7f810ce10 con 0x7fa7f81014a0 2023-12-17T12:08:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:26.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: Deploying daemon mds.f on smithi192 2023-12-17T12:08:26.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6834/4280622893,v1:172.21.15.141:6835/4280622893] up:boot 2023-12-17T12:08:26.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: fsmap 5 up:standby 2023-12-17T12:08:26.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:26 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-17T12:08:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.662+0000 7fa7f57fa700 1 -- 172.21.15.27:0/3547332855 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7fa7e8099be0 con 0x7fa7f81014a0 2023-12-17T12:08:26.663 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:26.663 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-12-17T12:08:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa7e40a1480 msgr2=0x7fa7e40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa7e40a1480 0x7fa7e40a3940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0011490 tx=0x7fa7e0009260 comp rx=0 tx=0).stop 2023-12-17T12:08:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7f81014a0 msgr2=0x7fa7f81973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7f81014a0 0x7fa7f81973d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800e970 tx=0x7fa7e800ec80 comp rx=0 tx=0).stop 2023-12-17T12:08:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 shutdown_connections 2023-12-17T12:08:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa7e40a1480 0x7fa7e40a3940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7f81014a0 0x7fa7f81973d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa7f8103030 0x7fa7f819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 --2- 172.21.15.27:0/3547332855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7f8102690 0x7fa7f8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:26.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.664+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 >> 172.21.15.27:0/3547332855 conn(0x7fa7f80fca10 msgr2=0x7fa7f8105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:26.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.665+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 shutdown_connections 2023-12-17T12:08:26.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:26.665+0000 7fa7fddff700 1 -- 172.21.15.27:0/3547332855 wait complete. 2023-12-17T12:08:26.666 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2023-12-17T12:08:26.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:26.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:26.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:26.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:26.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: Deploying daemon mds.f on smithi192 2023-12-17T12:08:26.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6834/4280622893,v1:172.21.15.141:6835/4280622893] up:boot 2023-12-17T12:08:26.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: fsmap 5 up:standby 2023-12-17T12:08:26.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:26 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-17T12:08:26.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:26.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:26.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:26.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:26.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:26.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: Deploying daemon mds.f on smithi192 2023-12-17T12:08:26.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6834/4280622893,v1:172.21.15.141:6835/4280622893] up:boot 2023-12-17T12:08:26.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: fsmap 5 up:standby 2023-12-17T12:08:26.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:26 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-17T12:08:27.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:27 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3547332855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:27.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:27 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3547332855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:27.811 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:08:27 smithi192 systemd[1]: Starting Ceph mds.f for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:27.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:27 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3547332855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:28.311 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:08:27 smithi192 bash[78267]: 58540c2b702a9cf30aa927f9c240bb83948885e7eedae202d8c9434695d16e9b 2023-12-17T12:08:28.311 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:08:27 smithi192 systemd[1]: Started Ceph mds.f for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:28.311 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:08:27 smithi192 ceph-mds[78305]: -- 172.21.15.192:0/302682558 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557746705520 con 0x5577466dcc00 2023-12-17T12:08:28.311 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:08:27 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-f[78282]: starting mds.f at 2023-12-17T12:08:28.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:28.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:28 smithi027 ceph-mon[36871]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:28.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.754 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.755 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:28.755 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:28.755 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:28 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:28.755 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:28 smithi192 ceph-mon[41899]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:28.755 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.756 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.756 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.756 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:28.756 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:28.756 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:28 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:28.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:28 smithi141 ceph-mon[41225]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:28.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:28.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:28.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:28.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:28 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:28.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 -- 172.21.15.27:0/3768457084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8075610 msgr2=0x7f66c8073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:28.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 --2- 172.21.15.27:0/3768457084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8075610 0x7f66c8073a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009990 tx=0x7f66b8009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:28.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 -- 172.21.15.27:0/3768457084 shutdown_connections 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 --2- 172.21.15.27:0/3768457084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8109360 0x7f66c810b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 --2- 172.21.15.27:0/3768457084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8075610 0x7f66c8073a60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 --2- 172.21.15.27:0/3768457084 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66c8074030 0x7f66c80744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 -- 172.21.15.27:0/3768457084 >> 172.21.15.27:0/3768457084 conn(0x7f66c80fca10 msgr2=0x7f66c80fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 -- 172.21.15.27:0/3768457084 shutdown_connections 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.905+0000 7f66cdbff700 1 -- 172.21.15.27:0/3768457084 wait complete. 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.906+0000 7f66cdbff700 1 Processor -- start 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.906+0000 7f66cdbff700 1 -- start start 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.906+0000 7f66cdbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8074030 0x7f66c8199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:28.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.906+0000 7f66cdbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8075610 0x7f66c8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.906+0000 7f66cdbff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66c8109360 0x7f66c81a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.906+0000 7f66cdbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81a1070 con 0x7f66c8074030 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.906+0000 7f66cdbff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81a11b0 con 0x7f66c8109360 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.906+0000 7f66cdbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81a1340 con 0x7f66c8075610 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8074030 0x7f66c8199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8075610 0x7f66c8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8074030 0x7f66c8199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34504/0 (socket says 172.21.15.27:34504) 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8075610 0x7f66c8199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:43470/0 (socket says 172.21.15.27:43470) 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 -- 172.21.15.27:0/471648118 learned_addr learned my addr 172.21.15.27:0/471648118 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c7fff700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66c8109360 0x7f66c81a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:28.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 -- 172.21.15.27:0/471648118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8075610 msgr2=0x7f66c8199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8075610 0x7f66c8199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 -- 172.21.15.27:0/471648118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66c8109360 msgr2=0x7f66c81a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66c8109360 0x7f66c81a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 -- 172.21.15.27:0/471648118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80095f0 con 0x7f66c8074030 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c7fff700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66c8109360 0x7f66c81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.907+0000 7f66c77fe700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8074030 0x7f66c8199540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66b8005740 tx=0x7f66b8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.908+0000 7f66c4ff9700 1 -- 172.21.15.27:0/471648118 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b801d070 con 0x7f66c8074030 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.908+0000 7f66c4ff9700 1 -- 172.21.15.27:0/471648118 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66b800beb0 con 0x7f66c8074030 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.908+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c81a15c0 con 0x7f66c8074030 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.908+0000 7f66c4ff9700 1 -- 172.21.15.27:0/471648118 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8021410 con 0x7f66c8074030 2023-12-17T12:08:28.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.908+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c81a19e0 con 0x7f66c8074030 2023-12-17T12:08:28.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.909+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c8108120 con 0x7f66c8074030 2023-12-17T12:08:28.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.912+0000 7f66c4ff9700 1 -- 172.21.15.27:0/471648118 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f66b800f720 con 0x7f66c8074030 2023-12-17T12:08:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.914+0000 7f66c4ff9700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66b00a1480 0x7f66b00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:28.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.914+0000 7f66c4ff9700 1 -- 172.21.15.27:0/471648118 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66b80ce0f0 con 0x7f66c8074030 2023-12-17T12:08:28.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.914+0000 7f66c6ffd700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66b00a1480 0x7f66b00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:28.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.915+0000 7f66c6ffd700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66b00a1480 0x7f66b00a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66c8075330 tx=0x7f66bc00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:28.916+0000 7f66c4ff9700 1 -- 172.21.15.27:0/471648118 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f66b8097be0 con 0x7f66c8074030 2023-12-17T12:08:29.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.179+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66c8068230 con 0x7f66c8074030 2023-12-17T12:08:29.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.180+0000 7f66c4ff9700 1 -- 172.21.15.27:0/471648118 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7f66b809b200 con 0x7f66c8074030 2023-12-17T12:08:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:29.181 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-12-17T12:08:29.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66b00a1480 msgr2=0x7f66b00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:29.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66b00a1480 0x7f66b00a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66c8075330 tx=0x7f66bc00b410 comp rx=0 tx=0).stop 2023-12-17T12:08:29.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8074030 msgr2=0x7f66c8199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:29.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8074030 0x7f66c8199540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66b8005740 tx=0x7f66b8005770 comp rx=0 tx=0).stop 2023-12-17T12:08:29.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 shutdown_connections 2023-12-17T12:08:29.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f66b00a1480 0x7f66b00a3940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:29.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f66c8109360 0x7f66c81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:29.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66c8074030 0x7f66c8199540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:29.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 --2- 172.21.15.27:0/471648118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8075610 0x7f66c8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:29.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 >> 172.21.15.27:0/471648118 conn(0x7f66c80fca10 msgr2=0x7f66c8105260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:29.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 shutdown_connections 2023-12-17T12:08:29.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:29.182+0000 7f66cdbff700 1 -- 172.21.15.27:0/471648118 wait complete. 2023-12-17T12:08:29.184 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-12-17T12:08:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:29 smithi027 ceph-mon[36871]: Deploying daemon mds.c on smithi192 2023-12-17T12:08:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:29 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.192:6834/2634479037,v1:172.21.15.192:6835/2634479037] up:boot 2023-12-17T12:08:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:29 smithi027 ceph-mon[36871]: fsmap 6 up:standby 2023-12-17T12:08:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:29 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-17T12:08:29.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:29 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/471648118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:29.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:29 smithi192 ceph-mon[41899]: Deploying daemon mds.c on smithi192 2023-12-17T12:08:29.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:29 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.192:6834/2634479037,v1:172.21.15.192:6835/2634479037] up:boot 2023-12-17T12:08:29.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:29 smithi192 ceph-mon[41899]: fsmap 6 up:standby 2023-12-17T12:08:29.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:29 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-17T12:08:29.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:29 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/471648118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:29.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:29 smithi141 ceph-mon[41225]: Deploying daemon mds.c on smithi192 2023-12-17T12:08:29.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:29 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.192:6834/2634479037,v1:172.21.15.192:6835/2634479037] up:boot 2023-12-17T12:08:29.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:29 smithi141 ceph-mon[41225]: fsmap 6 up:standby 2023-12-17T12:08:29.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:29 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-17T12:08:29.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:29 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/471648118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:29.977 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:08:29 smithi192 systemd[1]: Starting Ceph mds.c for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:30.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:30 smithi141 ceph-mon[41225]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:30.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:30 smithi192 ceph-mon[41899]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:30.811 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:08:30 smithi192 bash[78605]: baa644251777244e9843583bf51695e7ec4f4040b4db505382146100ec24f052 2023-12-17T12:08:30.811 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:08:30 smithi192 systemd[1]: Started Ceph mds.c for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:30.811 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:08:30 smithi192 ceph-mds[78643]: -- 172.21.15.192:0/1878116557 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eed0655520 con 0x55eed062cc00 2023-12-17T12:08:30.811 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:08:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-c[78620]: starting mds.c at 2023-12-17T12:08:30.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:30 smithi027 ceph-mon[36871]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:31.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:31.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 -- 172.21.15.27:0/622175506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6890103030 msgr2=0x7f68901098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:31.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 --2- 172.21.15.27:0/622175506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6890103030 0x7f68901098e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f688800b0d0 tx=0x7f688800b3e0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 -- 172.21.15.27:0/622175506 shutdown_connections 2023-12-17T12:08:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 --2- 172.21.15.27:0/622175506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6890103030 0x7f68901098e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 --2- 172.21.15.27:0/622175506 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f68901014a0 0x7f68901018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 --2- 172.21.15.27:0/622175506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6890102690 0x7f6890102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 -- 172.21.15.27:0/622175506 >> 172.21.15.27:0/622175506 conn(0x7f68900fca10 msgr2=0x7f68900fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 -- 172.21.15.27:0/622175506 shutdown_connections 2023-12-17T12:08:31.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.422+0000 7f6895fff700 1 -- 172.21.15.27:0/622175506 wait complete. 2023-12-17T12:08:31.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.423+0000 7f6895fff700 1 Processor -- start 2023-12-17T12:08:31.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.423+0000 7f6895fff700 1 -- start start 2023-12-17T12:08:31.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f6895fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68901014a0 0x7f68901995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:31.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f6895fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6890102690 0x7f6890199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f6895fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6890103030 0x7f68901a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f6895fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68901a1120 con 0x7f68901014a0 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f6895fff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68901a1260 con 0x7f6890103030 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f6895fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68901a13f0 con 0x7f6890102690 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68901014a0 0x7f68901995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68901014a0 0x7f68901995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34510/0 (socket says 172.21.15.27:34510) 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688f7fe700 1 -- 172.21.15.27:0/347413871 learned_addr learned my addr 172.21.15.27:0/347413871 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688ffff700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6890103030 0x7f68901a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688effd700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6890102690 0x7f6890199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688f7fe700 1 -- 172.21.15.27:0/347413871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6890102690 msgr2=0x7f6890199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688f7fe700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6890102690 0x7f6890199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688f7fe700 1 -- 172.21.15.27:0/347413871 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6890103030 msgr2=0x7f68901a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688f7fe700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6890103030 0x7f68901a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.424+0000 7f688f7fe700 1 -- 172.21.15.27:0/347413871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888009d00 con 0x7f68901014a0 2023-12-17T12:08:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.425+0000 7f688f7fe700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68901014a0 0x7f68901995f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f688000e970 tx=0x7f688000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.425+0000 7f688cff9700 1 -- 172.21.15.27:0/347413871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880009790 con 0x7f68901014a0 2023-12-17T12:08:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.425+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68901a16d0 con 0x7f68901014a0 2023-12-17T12:08:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.425+0000 7f688cff9700 1 -- 172.21.15.27:0/347413871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f688000eec0 con 0x7f68901014a0 2023-12-17T12:08:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.425+0000 7f688cff9700 1 -- 172.21.15.27:0/347413871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880003c10 con 0x7f68901014a0 2023-12-17T12:08:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.425+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68901a1be0 con 0x7f68901014a0 2023-12-17T12:08:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.426+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6890068230 con 0x7f68901014a0 2023-12-17T12:08:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.428+0000 7f688cff9700 1 -- 172.21.15.27:0/347413871 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6880003db0 con 0x7f68901014a0 2023-12-17T12:08:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.429+0000 7f688cff9700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f68780a1480 0x7f68780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.429+0000 7f688cff9700 1 -- 172.21.15.27:0/347413871 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6880014070 con 0x7f68901014a0 2023-12-17T12:08:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.430+0000 7f688effd700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f68780a1480 0x7f68780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.430+0000 7f688effd700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f68780a1480 0x7f68780a3940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6884009bf0 tx=0x7f6884009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:31.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.433+0000 7f688cff9700 1 -- 172.21.15.27:0/347413871 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f68800965c0 con 0x7f68901014a0 2023-12-17T12:08:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.693+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f689010cb60 con 0x7f68901014a0 2023-12-17T12:08:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.693+0000 7f688cff9700 1 -- 172.21.15.27:0/347413871 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6086 (secure 0 0 0) 0x7f6880099be0 con 0x7f68901014a0 2023-12-17T12:08:31.695 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:31.695 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-12-17T12:08:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f68780a1480 msgr2=0x7f68780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f68780a1480 0x7f68780a3940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6884009bf0 tx=0x7f6884009260 comp rx=0 tx=0).stop 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68901014a0 msgr2=0x7f68901995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68901014a0 0x7f68901995f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f688000e970 tx=0x7f688000ec80 comp rx=0 tx=0).stop 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 shutdown_connections 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f68780a1480 0x7f68780a3940 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6890103030 0x7f68901a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68901014a0 0x7f68901995f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 --2- 172.21.15.27:0/347413871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6890102690 0x7f6890199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 >> 172.21.15.27:0/347413871 conn(0x7f68900fca10 msgr2=0x7f6890107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.696+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 shutdown_connections 2023-12-17T12:08:31.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:31.697+0000 7f6895fff700 1 -- 172.21.15.27:0/347413871 wait complete. 2023-12-17T12:08:31.699 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-17T12:08:31.857 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:31.857 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:31.857 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:31.857 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:31.857 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:31.857 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:31.857 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: Deploying daemon mds.h on smithi141 2023-12-17T12:08:31.858 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.192:6836/1847164381,v1:172.21.15.192:6837/1847164381] up:boot 2023-12-17T12:08:31.858 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: fsmap 7 up:standby 2023-12-17T12:08:31.858 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:31 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: Deploying daemon mds.h on smithi141 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.192:6836/1847164381,v1:172.21.15.192:6837/1847164381] up:boot 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: fsmap 7 up:standby 2023-12-17T12:08:31.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:31 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: Deploying daemon mds.h on smithi141 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.192:6836/1847164381,v1:172.21.15.192:6837/1847164381] up:boot 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: fsmap 7 up:standby 2023-12-17T12:08:32.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:31 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-17T12:08:32.540 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:08:32 smithi141 systemd[1]: Starting Ceph mds.h for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:32 smithi027 ceph-mon[36871]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:32.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:32 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/347413871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:32.947 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:32 smithi141 ceph-mon[41225]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:32.947 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:32 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/347413871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:33.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:32 smithi192 ceph-mon[41899]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:33.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:32 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/347413871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:33.514 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:08:33 smithi141 bash[77080]: 5232468c94e3b1cec723cce7484adb3c175f7762f8daaa9b98bbe275cffb9fe3 2023-12-17T12:08:33.514 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:08:33 smithi141 systemd[1]: Started Ceph mds.h for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:33.514 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:08:33 smithi141 ceph-mds[77118]: -- 172.21.15.141:0/1343998973 <== mon.2 v2:172.21.15.141:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561881a67520 con 0x561881a3e800 2023-12-17T12:08:33.514 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:08:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-h[77095]: starting mds.h at 2023-12-17T12:08:33.718 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:33.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 -- 172.21.15.27:0/2372478949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c104890 msgr2=0x7f2a6c106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:33.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2372478949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c104890 0x7f2a6c106c80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009990 tx=0x7f2a5c009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:33.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 -- 172.21.15.27:0/2372478949 shutdown_connections 2023-12-17T12:08:33.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2372478949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c1071c0 0x7f2a6c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2372478949 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a6c101f60 0x7f2a6c104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2372478949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c104890 0x7f2a6c106c80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 -- 172.21.15.27:0/2372478949 >> 172.21.15.27:0/2372478949 conn(0x7f2a6c0fb8f0 msgr2=0x7f2a6c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 -- 172.21.15.27:0/2372478949 shutdown_connections 2023-12-17T12:08:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.939+0000 7f2a733ff700 1 -- 172.21.15.27:0/2372478949 wait complete. 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.940+0000 7f2a733ff700 1 Processor -- start 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.940+0000 7f2a733ff700 1 -- start start 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.940+0000 7f2a733ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c101f60 0x7f2a6c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.940+0000 7f2a733ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c104890 0x7f2a6c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.940+0000 7f2a733ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a6c1071c0 0x7f2a6c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.940+0000 7f2a733ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c1a1110 con 0x7f2a6c101f60 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.940+0000 7f2a733ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c1a1250 con 0x7f2a6c1071c0 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.940+0000 7f2a733ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c1a13e0 con 0x7f2a6c104890 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c101f60 0x7f2a6c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c101f60 0x7f2a6c1995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34534/0 (socket says 172.21.15.27:34534) 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 -- 172.21.15.27:0/2847561122 learned_addr learned my addr 172.21.15.27:0/2847561122 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7099a700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c104890 0x7f2a6c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7199c700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a6c1071c0 0x7f2a6c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 -- 172.21.15.27:0/2847561122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c104890 msgr2=0x7f2a6c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c104890 0x7f2a6c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 -- 172.21.15.27:0/2847561122 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a6c1071c0 msgr2=0x7f2a6c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a6c1071c0 0x7f2a6c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 -- 172.21.15.27:0/2847561122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c0095f0 con 0x7f2a6c101f60 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.941+0000 7f2a7119b700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c101f60 0x7f2a6c1995e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800d950 tx=0x7f2a6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.942+0000 7f2a627fc700 1 -- 172.21.15.27:0/2847561122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800f890 con 0x7f2a6c101f60 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.942+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c1a16c0 con 0x7f2a6c101f60 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.942+0000 7f2a627fc700 1 -- 172.21.15.27:0/2847561122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a6800fe80 con 0x7f2a6c101f60 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.942+0000 7f2a627fc700 1 -- 172.21.15.27:0/2847561122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800e3b0 con 0x7f2a6c101f60 2023-12-17T12:08:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.942+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c1a1bd0 con 0x7f2a6c101f60 2023-12-17T12:08:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.943+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6c108d60 con 0x7f2a6c101f60 2023-12-17T12:08:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.945+0000 7f2a627fc700 1 -- 172.21.15.27:0/2847561122 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2a68010460 con 0x7f2a6c101f60 2023-12-17T12:08:33.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.946+0000 7f2a627fc700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a580a1480 0x7f2a580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:33.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.946+0000 7f2a627fc700 1 -- 172.21.15.27:0/2847561122 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a680cc6e0 con 0x7f2a6c101f60 2023-12-17T12:08:33.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.946+0000 7f2a7099a700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a580a1480 0x7f2a580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:33.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.947+0000 7f2a7099a700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a580a1480 0x7f2a580a3940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c005910 tx=0x7f2a5c00b610 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:33.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:33.950+0000 7f2a627fc700 1 -- 172.21.15.27:0/2847561122 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2a68096140 con 0x7f2a6c101f60 2023-12-17T12:08:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.210+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a6c10ce90 con 0x7f2a6c101f60 2023-12-17T12:08:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.211+0000 7f2a627fc700 1 -- 172.21.15.27:0/2847561122 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6880 (secure 0 0 0) 0x7f2a68099760 con 0x7f2a6c101f60 2023-12-17T12:08:34.212 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:34.212 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-12-17T12:08:34.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a580a1480 msgr2=0x7f2a580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:34.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a580a1480 0x7f2a580a3940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c005910 tx=0x7f2a5c00b610 comp rx=0 tx=0).stop 2023-12-17T12:08:34.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c101f60 msgr2=0x7f2a6c1995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:34.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c101f60 0x7f2a6c1995e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800d950 tx=0x7f2a6800dc60 comp rx=0 tx=0).stop 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 shutdown_connections 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a580a1480 0x7f2a580a3940 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a6c1071c0 0x7f2a6c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a6c101f60 0x7f2a6c1995e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 --2- 172.21.15.27:0/2847561122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c104890 0x7f2a6c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 >> 172.21.15.27:0/2847561122 conn(0x7f2a6c0fb8f0 msgr2=0x7f2a6c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 shutdown_connections 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:34.214+0000 7f2a733ff700 1 -- 172.21.15.27:0/2847561122 wait complete. 2023-12-17T12:08:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2023-12-17T12:08:34.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:34.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: Deploying daemon mds.i on smithi192 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] up:boot 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: fsmap 8 up:standby 2023-12-17T12:08:34.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:34 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: Deploying daemon mds.i on smithi192 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] up:boot 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: fsmap 8 up:standby 2023-12-17T12:08:34.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:34 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-17T12:08:34.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:34.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:34.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:34.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:34.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:34.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: Deploying daemon mds.i on smithi192 2023-12-17T12:08:34.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] up:boot 2023-12-17T12:08:34.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: fsmap 8 up:standby 2023-12-17T12:08:34.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:34 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-17T12:08:35.210 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:35 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2847561122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:35.210 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:08:35 smithi192 systemd[1]: Starting Ceph mds.i for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:35.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:35 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2847561122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:35.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:35 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2847561122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:36.060 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:08:35 smithi192 bash[78947]: bb38209c02e3f5e8cd91c373ca8910623721f344137f34837ddb03d4589953d6 2023-12-17T12:08:36.061 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:08:35 smithi192 systemd[1]: Started Ceph mds.i for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:36.061 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:08:35 smithi192 ceph-mds[78985]: -- 172.21.15.192:0/2118551811 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564ee96ed520 con 0x564ee96c4800 2023-12-17T12:08:36.061 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:08:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-i[78962]: starting mds.i at 2023-12-17T12:08:36.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:36.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:36 smithi027 ceph-mon[36871]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:36.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:36.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:36.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:36 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:36.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 -- 172.21.15.27:0/3712328692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724107160 msgr2=0x7f47241095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:36.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 --2- 172.21.15.27:0/3712328692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724107160 0x7f47241095d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f471c00b0a0 tx=0x7f471c00b3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 -- 172.21.15.27:0/3712328692 shutdown_connections 2023-12-17T12:08:36.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 --2- 172.21.15.27:0/3712328692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724107160 0x7f47241095d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 --2- 172.21.15.27:0/3712328692 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4724101f00 0x7f47241042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 --2- 172.21.15.27:0/3712328692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4724104830 0x7f4724106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 -- 172.21.15.27:0/3712328692 >> 172.21.15.27:0/3712328692 conn(0x7f47240fb8f0 msgr2=0x7f47240fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:36.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 -- 172.21.15.27:0/3712328692 shutdown_connections 2023-12-17T12:08:36.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.465+0000 7f47293ff700 1 -- 172.21.15.27:0/3712328692 wait complete. 2023-12-17T12:08:36.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.466+0000 7f47293ff700 1 Processor -- start 2023-12-17T12:08:36.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.466+0000 7f47293ff700 1 -- start start 2023-12-17T12:08:36.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47293ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724101f00 0x7f4724101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:36.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47293ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4724104830 0x7f47240ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47293ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4724107160 0x7f47241003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47293ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724100b90 con 0x7f4724101f00 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47293ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724100d00 con 0x7f4724107160 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47293ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47241a14a0 con 0x7f4724104830 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f4722ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724101f00 0x7f4724101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f4722ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724101f00 0x7f4724101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36418/0 (socket says 172.21.15.27:36418) 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f4722ffd700 1 -- 172.21.15.27:0/1972551027 learned_addr learned my addr 172.21.15.27:0/1972551027 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47227fc700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4724104830 0x7f47240ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47237fe700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4724107160 0x7f47241003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f4722ffd700 1 -- 172.21.15.27:0/1972551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4724104830 msgr2=0x7f47240ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f4722ffd700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4724104830 0x7f47240ffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f4722ffd700 1 -- 172.21.15.27:0/1972551027 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4724107160 msgr2=0x7f47241003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f4722ffd700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4724107160 0x7f47241003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f4722ffd700 1 -- 172.21.15.27:0/1972551027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c009d00 con 0x7f4724101f00 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47237fe700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4724107160 0x7f47241003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:36.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.467+0000 7f47227fc700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4724104830 0x7f47240ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:08:36.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.468+0000 7f4722ffd700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724101f00 0x7f4724101820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f471400d950 tx=0x7f471400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:36.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.468+0000 7f470bfff700 1 -- 172.21.15.27:0/1972551027 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714009ae0 con 0x7f4724101f00 2023-12-17T12:08:36.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.468+0000 7f470bfff700 1 -- 172.21.15.27:0/1972551027 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4714009c40 con 0x7f4724101f00 2023-12-17T12:08:36.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.468+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47241a1730 con 0x7f4724101f00 2023-12-17T12:08:36.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.468+0000 7f470bfff700 1 -- 172.21.15.27:0/1972551027 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471400f620 con 0x7f4724101f00 2023-12-17T12:08:36.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.468+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47241a1c40 con 0x7f4724101f00 2023-12-17T12:08:36.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.469+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47240fd470 con 0x7f4724101f00 2023-12-17T12:08:36.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.471+0000 7f470bfff700 1 -- 172.21.15.27:0/1972551027 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4714010460 con 0x7f4724101f00 2023-12-17T12:08:36.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.473+0000 7f470bfff700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f470c0a1480 0x7f470c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:36.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.473+0000 7f470bfff700 1 -- 172.21.15.27:0/1972551027 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f47140cc8a0 con 0x7f4724101f00 2023-12-17T12:08:36.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.473+0000 7f47227fc700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f470c0a1480 0x7f470c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:36.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.474+0000 7f47227fc700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f470c0a1480 0x7f470c0a3940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f471800bc10 tx=0x7f471800b5f0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:36.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.475+0000 7f470bfff700 1 -- 172.21.15.27:0/1972551027 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4714096300 con 0x7f4724101f00 2023-12-17T12:08:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:36 smithi141 ceph-mon[41225]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:36.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:36 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:36 smithi192 ceph-mon[41899]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:36.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:36 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:36.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.755+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4724068230 con 0x7f4724101f00 2023-12-17T12:08:36.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.755+0000 7f470bfff700 1 -- 172.21.15.27:0/1972551027 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f4714099920 con 0x7f4724101f00 2023-12-17T12:08:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:36.757 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f470c0a1480 msgr2=0x7f470c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f470c0a1480 0x7f470c0a3940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f471800bc10 tx=0x7f471800b5f0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724101f00 msgr2=0x7f4724101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724101f00 0x7f4724101820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f471400d950 tx=0x7f471400dd10 comp rx=0 tx=0).stop 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 shutdown_connections 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f470c0a1480 0x7f470c0a3940 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4724107160 0x7f47241003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4724101f00 0x7f4724101820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 --2- 172.21.15.27:0/1972551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4724104830 0x7f47240ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 >> 172.21.15.27:0/1972551027 conn(0x7f47240fb8f0 msgr2=0x7f4724102920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 shutdown_connections 2023-12-17T12:08:36.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:36.758+0000 7f47293ff700 1 -- 172.21.15.27:0/1972551027 wait complete. 2023-12-17T12:08:36.761 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2023-12-17T12:08:37.306 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:37 smithi027 ceph-mon[36871]: Deploying daemon mds.g on smithi027 2023-12-17T12:08:37.306 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:37 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] up:boot 2023-12-17T12:08:37.306 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:37 smithi027 ceph-mon[36871]: fsmap 9 up:standby 2023-12-17T12:08:37.306 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:37 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-17T12:08:37.306 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:37 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1972551027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:37.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:37 smithi141 ceph-mon[41225]: Deploying daemon mds.g on smithi027 2023-12-17T12:08:37.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:37 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] up:boot 2023-12-17T12:08:37.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:37 smithi141 ceph-mon[41225]: fsmap 9 up:standby 2023-12-17T12:08:37.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:37 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-17T12:08:37.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:37 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1972551027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:37.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:37 smithi192 ceph-mon[41899]: Deploying daemon mds.g on smithi027 2023-12-17T12:08:37.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:37 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] up:boot 2023-12-17T12:08:37.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:37 smithi192 ceph-mon[41899]: fsmap 9 up:standby 2023-12-17T12:08:37.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:37 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-17T12:08:37.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:37 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1972551027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:38.162 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:08:37 smithi027 systemd[1]: Starting Ceph mds.g for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:38.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:38 smithi027 ceph-mon[36871]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:38.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:38 smithi141 ceph-mon[41225]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:38.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:38 smithi192 ceph-mon[41899]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:38.781 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:38.845 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:08:38 smithi027 ceph-mds[88361]: -- 172.21.15.27:0/2819232203 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b80ba9520 con 0x564b80b80800 2023-12-17T12:08:38.846 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:08:38 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-g[88338]: starting mds.g at 2023-12-17T12:08:38.846 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:08:38 smithi027 bash[88323]: e841b8463b72ec56e5ba1ee691c965b8fbb7e46a2bfb85a0fba39e0928ea9987 2023-12-17T12:08:38.846 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:08:38 smithi027 systemd[1]: Started Ceph mds.g for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:38.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 -- 172.21.15.27:0/1098691744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 msgr2=0x7f73d0109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:38.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 --2- 172.21.15.27:0/1098691744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 0x7f73d0109880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73c800b0d0 tx=0x7f73c800b3e0 comp rx=0 tx=0).stop 2023-12-17T12:08:38.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 -- 172.21.15.27:0/1098691744 shutdown_connections 2023-12-17T12:08:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 --2- 172.21.15.27:0/1098691744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 0x7f73d0109880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 --2- 172.21.15.27:0/1098691744 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f73d0101440 0x7f73d0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 --2- 172.21.15.27:0/1098691744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73d0102630 0x7f73d0102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 -- 172.21.15.27:0/1098691744 >> 172.21.15.27:0/1098691744 conn(0x7f73d00fc990 msgr2=0x7f73d00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 -- 172.21.15.27:0/1098691744 shutdown_connections 2023-12-17T12:08:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.996+0000 7f73d73ff700 1 -- 172.21.15.27:0/1098691744 wait complete. 2023-12-17T12:08:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.997+0000 7f73d73ff700 1 Processor -- start 2023-12-17T12:08:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d73ff700 1 -- start start 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d73ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f73d0101440 0x7f73d0199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d73ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73d0102630 0x7f73d0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d73ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 0x7f73d01a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d73ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01a10a0 con 0x7f73d0102fd0 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d73ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01a11e0 con 0x7f73d0101440 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d73ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01a1370 con 0x7f73d0102630 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d499a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73d0102630 0x7f73d0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d599c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 0x7f73d01a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d519b700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f73d0101440 0x7f73d0199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d599c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 0x7f73d01a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36452/0 (socket says 172.21.15.27:36452) 2023-12-17T12:08:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d599c700 1 -- 172.21.15.27:0/3044476974 learned_addr learned my addr 172.21.15.27:0/3044476974 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.998+0000 7f73d499a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73d0102630 0x7f73d0199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:58312/0 (socket says 172.21.15.27:58312) 2023-12-17T12:08:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.999+0000 7f73d519b700 1 -- 172.21.15.27:0/3044476974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73d0102630 msgr2=0x7f73d0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.999+0000 7f73d519b700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73d0102630 0x7f73d0199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.999+0000 7f73d519b700 1 -- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 msgr2=0x7f73d01a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.999+0000 7f73d519b700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 0x7f73d01a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.999+0000 7f73d519b700 1 -- 172.21.15.27:0/3044476974 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c8009d00 con 0x7f73d0101440 2023-12-17T12:08:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.999+0000 7f73d599c700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 0x7f73d01a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:38.999+0000 7f73d519b700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f73d0101440 0x7f73d0199570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00e9c0 tx=0x7f73cc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.000+0000 7f73c67fc700 1 -- 172.21.15.27:0/3044476974 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc004220 con 0x7f73d0101440 2023-12-17T12:08:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.000+0000 7f73c67fc700 1 -- 172.21.15.27:0/3044476974 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73cc004380 con 0x7f73d0101440 2023-12-17T12:08:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.000+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d01a1650 con 0x7f73d0101440 2023-12-17T12:08:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.000+0000 7f73c67fc700 1 -- 172.21.15.27:0/3044476974 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc003b40 con 0x7f73d0101440 2023-12-17T12:08:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.000+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d01a1bb0 con 0x7f73d0101440 2023-12-17T12:08:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.001+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d00681c0 con 0x7f73d0101440 2023-12-17T12:08:39.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.004+0000 7f73c67fc700 1 -- 172.21.15.27:0/3044476974 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f73cc003ca0 con 0x7f73d0101440 2023-12-17T12:08:39.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.005+0000 7f73c67fc700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73bc0a1500 0x7f73bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:39.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.005+0000 7f73c67fc700 1 -- 172.21.15.27:0/3044476974 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73cc014070 con 0x7f73d0101440 2023-12-17T12:08:39.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.005+0000 7f73d499a700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73bc0a1500 0x7f73bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:39.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.006+0000 7f73d499a700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73bc0a1500 0x7f73bc0a39c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009c60 tx=0x7f73c0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.008+0000 7f73c67fc700 1 -- 172.21.15.27:0/3044476974 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f73cc096910 con 0x7f73d0101440 2023-12-17T12:08:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.282+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73d010cdc0 con 0x7f73d0101440 2023-12-17T12:08:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.283+0000 7f73c67fc700 1 -- 172.21.15.27:0/3044476974 <== mon.2 v2:172.21.15.141:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7f73cc017e90 con 0x7f73d0101440 2023-12-17T12:08:39.284 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:39.284 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-17T12:08:39.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73bc0a1500 msgr2=0x7f73bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:39.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73bc0a1500 0x7f73bc0a39c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009c60 tx=0x7f73c0009260 comp rx=0 tx=0).stop 2023-12-17T12:08:39.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f73d0101440 msgr2=0x7f73d0199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:39.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f73d0101440 0x7f73d0199570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00e9c0 tx=0x7f73cc00ecd0 comp rx=0 tx=0).stop 2023-12-17T12:08:39.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 shutdown_connections 2023-12-17T12:08:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f73bc0a1500 0x7f73bc0a39c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73d0102fd0 0x7f73d01a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f73d0101440 0x7f73d0199570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 --2- 172.21.15.27:0/3044476974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73d0102630 0x7f73d0199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 >> 172.21.15.27:0/3044476974 conn(0x7f73d00fc990 msgr2=0x7f73d0073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 shutdown_connections 2023-12-17T12:08:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:39.286+0000 7f73d73ff700 1 -- 172.21.15.27:0/3044476974 wait complete. 2023-12-17T12:08:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 11 2023-12-17T12:08:39.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: Deploying daemon mds.a on smithi027 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.27:6838/1099412442,v1:172.21.15.27:6839/1099412442] up:boot 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: fsmap 10 up:standby 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-17T12:08:39.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:39 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3044476974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:39.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: Deploying daemon mds.a on smithi027 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.27:6838/1099412442,v1:172.21.15.27:6839/1099412442] up:boot 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: fsmap 10 up:standby 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-17T12:08:39.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:39 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3044476974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: Deploying daemon mds.a on smithi027 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.27:6838/1099412442,v1:172.21.15.27:6839/1099412442] up:boot 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: fsmap 10 up:standby 2023-12-17T12:08:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-17T12:08:40.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:39 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3044476974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:40.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:40 smithi027 ceph-mon[36871]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:40.912 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:08:40 smithi027 systemd[1]: Starting Ceph mds.a for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:41.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:40 smithi141 ceph-mon[41225]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:41.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:40 smithi192 ceph-mon[41899]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:41.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:41.377 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:08:41 smithi027 ceph-mds[88744]: -- 172.21.15.27:0/963969465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e8a6203520 con 0x55e8a61dac00 2023-12-17T12:08:41.378 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:08:41 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-a[88721]: starting mds.a at 2023-12-17T12:08:41.378 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:08:41 smithi027 bash[88706]: 24d0e0dfcacb2d3dbe6994c13d2d05e350ba31848fa424356efaa7ff05789b1d 2023-12-17T12:08:41.378 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:08:41 smithi027 systemd[1]: Started Ceph mds.a for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:41.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 -- 172.21.15.27:0/1599378498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3448103030 msgr2=0x7f34481098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:41.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 --2- 172.21.15.27:0/1599378498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3448103030 0x7f34481098e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f344000b0d0 tx=0x7f344000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 -- 172.21.15.27:0/1599378498 shutdown_connections 2023-12-17T12:08:41.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 --2- 172.21.15.27:0/1599378498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34481014a0 0x7f34481018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 --2- 172.21.15.27:0/1599378498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3448103030 0x7f34481098e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 --2- 172.21.15.27:0/1599378498 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3448102690 0x7f3448102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 -- 172.21.15.27:0/1599378498 >> 172.21.15.27:0/1599378498 conn(0x7f34480fca10 msgr2=0x7f34480fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:41.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 -- 172.21.15.27:0/1599378498 shutdown_connections 2023-12-17T12:08:41.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.528+0000 7f344e5ff700 1 -- 172.21.15.27:0/1599378498 wait complete. 2023-12-17T12:08:41.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.529+0000 7f344e5ff700 1 Processor -- start 2023-12-17T12:08:41.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.529+0000 7f344e5ff700 1 -- start start 2023-12-17T12:08:41.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f344e5ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34481014a0 0x7f3448075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:41.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f344e5ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3448102690 0x7f34480739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f344e5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3448103030 0x7f3448073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f344e5ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34480746a0 con 0x7f34481014a0 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f344e5ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3448074810 con 0x7f3448102690 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f344e5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34481a1490 con 0x7f3448103030 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f3447fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34481014a0 0x7f3448075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f3447fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34481014a0 0x7f3448075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36498/0 (socket says 172.21.15.27:36498) 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f3447fff700 1 -- 172.21.15.27:0/4255109498 learned_addr learned my addr 172.21.15.27:0/4255109498 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f34477fe700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3448102690 0x7f34480739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f344cb9c700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3448103030 0x7f3448073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f3447fff700 1 -- 172.21.15.27:0/4255109498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3448103030 msgr2=0x7f3448073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f3447fff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3448103030 0x7f3448073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f3447fff700 1 -- 172.21.15.27:0/4255109498 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3448102690 msgr2=0x7f34480739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f3447fff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3448102690 0x7f34480739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.530+0000 7f3447fff700 1 -- 172.21.15.27:0/4255109498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3440009d00 con 0x7f34481014a0 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.531+0000 7f3447fff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34481014a0 0x7f3448075330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f343800c970 tx=0x7f343800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.531+0000 7f34457fa700 1 -- 172.21.15.27:0/4255109498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438004220 con 0x7f34481014a0 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.531+0000 7f34457fa700 1 -- 172.21.15.27:0/4255109498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f343800ce60 con 0x7f34481014a0 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.531+0000 7f34457fa700 1 -- 172.21.15.27:0/4255109498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438003b60 con 0x7f34481014a0 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.531+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34481a1720 con 0x7f34481014a0 2023-12-17T12:08:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.531+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34481a1be0 con 0x7f34481014a0 2023-12-17T12:08:41.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.532+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3448108970 con 0x7f34481014a0 2023-12-17T12:08:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.534+0000 7f34457fa700 1 -- 172.21.15.27:0/4255109498 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3438003cc0 con 0x7f34481014a0 2023-12-17T12:08:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.535+0000 7f34457fa700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34300a1480 0x7f34300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.536+0000 7f34457fa700 1 -- 172.21.15.27:0/4255109498 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f34380cc470 con 0x7f34481014a0 2023-12-17T12:08:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.536+0000 7f34477fe700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34300a1480 0x7f34300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.536+0000 7f34477fe700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34300a1480 0x7f34300a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f343c009c60 tx=0x7f343c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.540+0000 7f34457fa700 1 -- 172.21.15.27:0/4255109498 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3438095ed0 con 0x7f34481014a0 2023-12-17T12:08:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.801+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3448068230 con 0x7f34481014a0 2023-12-17T12:08:41.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.802+0000 7f34457fa700 1 -- 172.21.15.27:0/4255109498 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9257 (secure 0 0 0) 0x7f34380994f0 con 0x7f34481014a0 2023-12-17T12:08:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:41.804 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3551652066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3551652066},{"type":"v1","addr":"172.21.15.27:6841","nonce":3551652066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34300a1480 msgr2=0x7f34300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34300a1480 0x7f34300a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f343c009c60 tx=0x7f343c009260 comp rx=0 tx=0).stop 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34481014a0 msgr2=0x7f3448075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34481014a0 0x7f3448075330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f343800c970 tx=0x7f343800cc80 comp rx=0 tx=0).stop 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 shutdown_connections 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34300a1480 0x7f34300a3940 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3448103030 0x7f3448073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34481014a0 0x7f3448075330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 --2- 172.21.15.27:0/4255109498 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3448102690 0x7f34480739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 >> 172.21.15.27:0/4255109498 conn(0x7f34480fca10 msgr2=0x7f3448105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 shutdown_connections 2023-12-17T12:08:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:41.805+0000 7f344e5ff700 1 -- 172.21.15.27:0/4255109498 wait complete. 2023-12-17T12:08:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 12 2023-12-17T12:08:42.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:42.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:42.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:42.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:42.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: Deploying daemon mds.b on smithi141 2023-12-17T12:08:42.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.27:6840/3551652066,v1:172.21.15.27:6841/3551652066] up:boot 2023-12-17T12:08:42.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: fsmap 11 up:standby 2023-12-17T12:08:42.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-17T12:08:42.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:42 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4255109498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: Deploying daemon mds.b on smithi141 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.27:6840/3551652066,v1:172.21.15.27:6841/3551652066] up:boot 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: fsmap 11 up:standby 2023-12-17T12:08:42.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-17T12:08:42.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:42 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4255109498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: Deploying daemon mds.b on smithi141 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.27:6840/3551652066,v1:172.21.15.27:6841/3551652066] up:boot 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: fsmap 11 up:standby 2023-12-17T12:08:42.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-17T12:08:42.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:42 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4255109498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:43.496 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:08:43 smithi141 systemd[1]: Starting Ceph mds.b for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:08:43.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:44.014 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:08:43 smithi141 bash[77421]: 9145a4c3781e9ead02f73d91c88f7cd1d72411c520caaabfbed67972987dbc87 2023-12-17T12:08:44.014 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:08:43 smithi141 systemd[1]: Started Ceph mds.b for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:08:44.014 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:08:43 smithi141 ceph-mds[77459]: -- 172.21.15.141:0/3392086518 <== mon.2 v2:172.21.15.141:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560699e39520 con 0x560699e11000 2023-12-17T12:08:44.014 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:08:43 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-b[77436]: starting mds.b at 2023-12-17T12:08:44.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/1449085011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 msgr2=0x7f8b481098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/1449085011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 0x7f8b481098e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000b0d0 tx=0x7f8b4000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/1449085011 shutdown_connections 2023-12-17T12:08:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/1449085011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 0x7f8b481098e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/1449085011 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8b481014a0 0x7f8b481018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/1449085011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b48102690 0x7f8b48102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/1449085011 >> 172.21.15.27:0/1449085011 conn(0x7f8b480fca10 msgr2=0x7f8b480fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/1449085011 shutdown_connections 2023-12-17T12:08:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.037+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/1449085011 wait complete. 2023-12-17T12:08:44.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.038+0000 7f8b4e9ff700 1 Processor -- start 2023-12-17T12:08:44.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.038+0000 7f8b4e9ff700 1 -- start start 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.038+0000 7f8b4e9ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8b481014a0 0x7f8b48199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4e9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b48102690 0x7f8b48199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4e9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 0x7f8b481a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4e9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481a1160 con 0x7f8b48103030 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4e9ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481a12a0 con 0x7f8b481014a0 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4e9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481a1430 con 0x7f8b48102690 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4cf9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 0x7f8b481a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b47fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8b481014a0 0x7f8b48199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b3ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b48102690 0x7f8b48199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b47fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8b481014a0 0x7f8b48199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:36428/0 (socket says 172.21.15.27:36428) 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4cf9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 0x7f8b481a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36526/0 (socket says 172.21.15.27:36526) 2023-12-17T12:08:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4cf9c700 1 -- 172.21.15.27:0/99157398 learned_addr learned my addr 172.21.15.27:0/99157398 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4cf9c700 1 -- 172.21.15.27:0/99157398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b48102690 msgr2=0x7f8b48199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4cf9c700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b48102690 0x7f8b48199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4cf9c700 1 -- 172.21.15.27:0/99157398 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8b481014a0 msgr2=0x7f8b48199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4cf9c700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8b481014a0 0x7f8b48199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.039+0000 7f8b4cf9c700 1 -- 172.21.15.27:0/99157398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40009d00 con 0x7f8b48103030 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.040+0000 7f8b4cf9c700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 0x7f8b481a0b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b400079c0 tx=0x7f8b4000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.040+0000 7f8b45ffb700 1 -- 172.21.15.27:0/99157398 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000e040 con 0x7f8b48103030 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.040+0000 7f8b45ffb700 1 -- 172.21.15.27:0/99157398 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b4000ceb0 con 0x7f8b48103030 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.040+0000 7f8b45ffb700 1 -- 172.21.15.27:0/99157398 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4001b6b0 con 0x7f8b48103030 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.040+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b481a16b0 con 0x7f8b48103030 2023-12-17T12:08:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.040+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b481a1ae0 con 0x7f8b48103030 2023-12-17T12:08:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.041+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b48068230 con 0x7f8b48103030 2023-12-17T12:08:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.043+0000 7f8b45ffb700 1 -- 172.21.15.27:0/99157398 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8b4001d480 con 0x7f8b48103030 2023-12-17T12:08:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.044+0000 7f8b45ffb700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8b340a1500 0x7f8b340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.044+0000 7f8b45ffb700 1 -- 172.21.15.27:0/99157398 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b400ce130 con 0x7f8b48103030 2023-12-17T12:08:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.045+0000 7f8b47fff700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8b340a1500 0x7f8b340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.045+0000 7f8b47fff700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8b340a1500 0x7f8b340a39c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b481024f0 tx=0x7f8b38009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.049+0000 7f8b45ffb700 1 -- 172.21.15.27:0/99157398 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8b40097c20 con 0x7f8b48103030 2023-12-17T12:08:44.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.317+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b4810cb70 con 0x7f8b48103030 2023-12-17T12:08:44.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.318+0000 7f8b45ffb700 1 -- 172.21.15.27:0/99157398 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10052 (secure 0 0 0) 0x7f8b4009b240 con 0x7f8b48103030 2023-12-17T12:08:44.319 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:44.319 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3551652066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3551652066},{"type":"v1","addr":"172.21.15.27:6841","nonce":3551652066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24509,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6839/3282544216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6838","nonce":3282544216},{"type":"v1","addr":"172.21.15.141:6839","nonce":3282544216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8b340a1500 msgr2=0x7f8b340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8b340a1500 0x7f8b340a39c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b481024f0 tx=0x7f8b38009260 comp rx=0 tx=0).stop 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 msgr2=0x7f8b481a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 0x7f8b481a0b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b400079c0 tx=0x7f8b4000cab0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 shutdown_connections 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8b340a1500 0x7f8b340a39c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b48103030 0x7f8b481a0b90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8b481014a0 0x7f8b48199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 --2- 172.21.15.27:0/99157398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b48102690 0x7f8b48199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.320+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 >> 172.21.15.27:0/99157398 conn(0x7f8b480fca10 msgr2=0x7f8b48073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:44.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.321+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 shutdown_connections 2023-12-17T12:08:44.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.321+0000 7f8b4e9ff700 1 -- 172.21.15.27:0/99157398 wait complete. 2023-12-17T12:08:44.323 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2023-12-17T12:08:44.342 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:08:44.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.554+0000 7f1a519ff700 1 -- 172.21.15.27:0/1463434979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c10da70 msgr2=0x7f1a4c10fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.554+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1463434979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c10da70 0x7f1a4c10fe60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400b0a0 tx=0x7f1a4400b3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.555+0000 7f1a519ff700 1 -- 172.21.15.27:0/1463434979 shutdown_connections 2023-12-17T12:08:44.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.555+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1463434979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c10da70 0x7f1a4c10fe60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.555+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1463434979 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1a4c075610 0x7f1a4c073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.555+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1463434979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c074030 0x7f1a4c0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.555+0000 7f1a519ff700 1 -- 172.21.15.27:0/1463434979 >> 172.21.15.27:0/1463434979 conn(0x7f1a4c0fca10 msgr2=0x7f1a4c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:44.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.555+0000 7f1a519ff700 1 -- 172.21.15.27:0/1463434979 shutdown_connections 2023-12-17T12:08:44.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.555+0000 7f1a519ff700 1 -- 172.21.15.27:0/1463434979 wait complete. 2023-12-17T12:08:44.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.556+0000 7f1a519ff700 1 Processor -- start 2023-12-17T12:08:44.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.556+0000 7f1a519ff700 1 -- start start 2023-12-17T12:08:44.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a519ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c074030 0x7f1a4c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:44.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a519ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c075610 0x7f1a4c19df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:44.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a519ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1a4c10da70 0x7f1a4c1a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:44.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a519ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1a5530 con 0x7f1a4c075610 2023-12-17T12:08:44.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a519ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1a5670 con 0x7f1a4c10da70 2023-12-17T12:08:44.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a519ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1a5800 con 0x7f1a4c074030 2023-12-17T12:08:44.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a4a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c075610 0x7f1a4c19df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:44.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a4affd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c074030 0x7f1a4c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:44.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a4affd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c074030 0x7f1a4c19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:58406/0 (socket says 172.21.15.27:58406) 2023-12-17T12:08:44.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a4affd700 1 -- 172.21.15.27:0/1137177272 learned_addr learned my addr 172.21.15.27:0/1137177272 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:44.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.557+0000 7f1a4b7fe700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1a4c10da70 0x7f1a4c1a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a4affd700 1 -- 172.21.15.27:0/1137177272 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1a4c10da70 msgr2=0x7f1a4c1a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a4affd700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1a4c10da70 0x7f1a4c1a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a4affd700 1 -- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c075610 msgr2=0x7f1a4c19df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a4affd700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c075610 0x7f1a4c19df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a4affd700 1 -- 172.21.15.27:0/1137177272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c009520 con 0x7f1a4c074030 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a4a7fc700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c075610 0x7f1a4c19df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a4affd700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c074030 0x7f1a4c19da00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00eb30 tx=0x7f1a3c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a509fd700 1 -- 172.21.15.27:0/1137177272 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c009880 con 0x7f1a4c074030 2023-12-17T12:08:44.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a509fd700 1 -- 172.21.15.27:0/1137177272 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a3c0044b0 con 0x7f1a4c074030 2023-12-17T12:08:44.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a509fd700 1 -- 172.21.15.27:0/1137177272 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c003ce0 con 0x7f1a4c074030 2023-12-17T12:08:44.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.558+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a44009d00 con 0x7f1a4c074030 2023-12-17T12:08:44.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.559+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c1a5e60 con 0x7f1a4c074030 2023-12-17T12:08:44.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.560+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c103cd0 con 0x7f1a4c074030 2023-12-17T12:08:44.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.562+0000 7f1a509fd700 1 -- 172.21.15.27:0/1137177272 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1a3c003e40 con 0x7f1a4c074030 2023-12-17T12:08:44.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.564+0000 7f1a509fd700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a340a1500 0x7f1a340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:44.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.564+0000 7f1a509fd700 1 -- 172.21.15.27:0/1137177272 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1a3c014070 con 0x7f1a4c074030 2023-12-17T12:08:44.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.564+0000 7f1a4a7fc700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a340a1500 0x7f1a340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:44.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.565+0000 7f1a4a7fc700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a340a1500 0x7f1a340a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c075330 tx=0x7f1a40009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.567+0000 7f1a509fd700 1 -- 172.21.15.27:0/1137177272 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1a3c0967c0 con 0x7f1a4c074030 2023-12-17T12:08:44.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.809+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a4c068230 con 0x7f1a4c074030 2023-12-17T12:08:44.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.810+0000 7f1a509fd700 1 -- 172.21.15.27:0/1137177272 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18433 (secure 0 0 0) 0x7f1a4c068230 con 0x7f1a4c074030 2023-12-17T12:08:44.812 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:44.812 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":78,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:08:03.382807+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:08:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.812+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a340a1500 msgr2=0x7f1a340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a340a1500 0x7f1a340a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c075330 tx=0x7f1a40009260 comp rx=0 tx=0).stop 2023-12-17T12:08:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c074030 msgr2=0x7f1a4c19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c074030 0x7f1a4c19da00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00eb30 tx=0x7f1a3c00ee40 comp rx=0 tx=0).stop 2023-12-17T12:08:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 shutdown_connections 2023-12-17T12:08:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1a340a1500 0x7f1a340a39c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c074030 0x7f1a4c19da00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1a4c10da70 0x7f1a4c1a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 --2- 172.21.15.27:0/1137177272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c075610 0x7f1a4c19df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 >> 172.21.15.27:0/1137177272 conn(0x7f1a4c0fca10 msgr2=0x7f1a4c100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 shutdown_connections 2023-12-17T12:08:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:44.813+0000 7f1a519ff700 1 -- 172.21.15.27:0/1137177272 wait complete. 2023-12-17T12:08:44.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-17T12:04:41.530587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-17T12:08:44.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] up:boot 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: fsmap 12 up:standby 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-17T12:08:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:44 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/99157398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 -- 172.21.15.27:0/3206363020 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8bd4074030 msgr2=0x7f8bd40744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/3206363020 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8bd4074030 0x7f8bd40744b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009990 tx=0x7f8bc8009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 -- 172.21.15.27:0/3206363020 shutdown_connections 2023-12-17T12:08:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/3206363020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd410db70 0x7f8bd410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/3206363020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4075610 0x7f8bd4073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/3206363020 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8bd4074030 0x7f8bd40744b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 -- 172.21.15.27:0/3206363020 >> 172.21.15.27:0/3206363020 conn(0x7f8bd40fcb10 msgr2=0x7f8bd40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 -- 172.21.15.27:0/3206363020 shutdown_connections 2023-12-17T12:08:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.053+0000 7f8bd91ff700 1 -- 172.21.15.27:0/3206363020 wait complete. 2023-12-17T12:08:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.054+0000 7f8bd91ff700 1 Processor -- start 2023-12-17T12:08:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd91ff700 1 -- start start 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd91ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd4074030 0x7f8bd419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd91ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4075610 0x7f8bd419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd91ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8bd410db70 0x7f8bd41a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd91ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd41a5550 con 0x7f8bd4075610 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd91ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd41a5690 con 0x7f8bd410db70 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd91ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd41a5820 con 0x7f8bd4074030 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd4074030 0x7f8bd419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd4074030 0x7f8bd419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:58422/0 (socket says 172.21.15.27:58422) 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd2d9d700 1 -- 172.21.15.27:0/1238172983 learned_addr learned my addr 172.21.15.27:0/1238172983 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd259c700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4075610 0x7f8bd419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.055+0000 7f8bd359e700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8bd410db70 0x7f8bd41a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bd2d9d700 1 -- 172.21.15.27:0/1238172983 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8bd410db70 msgr2=0x7f8bd41a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bd2d9d700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8bd410db70 0x7f8bd41a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bd2d9d700 1 -- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4075610 msgr2=0x7f8bd419df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bd2d9d700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4075610 0x7f8bd419df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bd2d9d700 1 -- 172.21.15.27:0/1238172983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4009520 con 0x7f8bd4074030 2023-12-17T12:08:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bd2d9d700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd4074030 0x7f8bd419da20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400eb30 tx=0x7f8bc400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bbbfff700 1 -- 172.21.15.27:0/1238172983 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4009880 con 0x7f8bd4074030 2023-12-17T12:08:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc80095f0 con 0x7f8bd4074030 2023-12-17T12:08:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bbbfff700 1 -- 172.21.15.27:0/1238172983 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bc40044b0 con 0x7f8bd4074030 2023-12-17T12:08:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bbbfff700 1 -- 172.21.15.27:0/1238172983 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4003ce0 con 0x7f8bd4074030 2023-12-17T12:08:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.056+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd41a5ed0 con 0x7f8bd4074030 2023-12-17T12:08:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.057+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd4103dd0 con 0x7f8bd4074030 2023-12-17T12:08:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.059+0000 7f8bbbfff700 1 -- 172.21.15.27:0/1238172983 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8bc4003e40 con 0x7f8bd4074030 2023-12-17T12:08:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:45.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] up:boot 2023-12-17T12:08:45.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: fsmap 12 up:standby 2023-12-17T12:08:45.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-17T12:08:45.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:44 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/99157398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.061+0000 7f8bbbfff700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8bbc0a1480 0x7f8bbc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.061+0000 7f8bbbfff700 1 -- 172.21.15.27:0/1238172983 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8bc4014070 con 0x7f8bd4074030 2023-12-17T12:08:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.061+0000 7f8bd259c700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8bbc0a1480 0x7f8bbc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:45.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.062+0000 7f8bd259c700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8bbc0a1480 0x7f8bbc0a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8006010 tx=0x7f8bc800b610 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:45.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.064+0000 7f8bbbfff700 1 -- 172.21.15.27:0/1238172983 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8bc4096840 con 0x7f8bd4074030 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] up:boot 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: fsmap 12 up:standby 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-17T12:08:45.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:44 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/99157398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:45.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.284+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bd41a59e0 con 0x7f8bd4074030 2023-12-17T12:08:45.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.285+0000 7f8bbbfff700 1 -- 172.21.15.27:0/1238172983 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bc4099e60 con 0x7f8bd4074030 2023-12-17T12:08:45.287 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-17T12:08:45.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8bbc0a1480 msgr2=0x7f8bbc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8bbc0a1480 0x7f8bbc0a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8006010 tx=0x7f8bc800b610 comp rx=0 tx=0).stop 2023-12-17T12:08:45.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd4074030 msgr2=0x7f8bd419da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd4074030 0x7f8bd419da20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400eb30 tx=0x7f8bc400ee40 comp rx=0 tx=0).stop 2023-12-17T12:08:45.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 shutdown_connections 2023-12-17T12:08:45.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8bbc0a1480 0x7f8bbc0a3940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd4074030 0x7f8bd419da20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8bd410db70 0x7f8bd41a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 --2- 172.21.15.27:0/1238172983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd4075610 0x7f8bd419df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 >> 172.21.15.27:0/1238172983 conn(0x7f8bd40fcb10 msgr2=0x7f8bd4100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:45.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 shutdown_connections 2023-12-17T12:08:45.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.288+0000 7f8bd91ff700 1 -- 172.21.15.27:0/1238172983 wait complete. 2023-12-17T12:08:45.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-17T12:08:45.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 -- 172.21.15.27:0/2723563607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64102680 msgr2=0x7fee64102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 --2- 172.21.15.27:0/2723563607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64102680 0x7fee64102ae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee580099c0 tx=0x7fee58009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 -- 172.21.15.27:0/2723563607 shutdown_connections 2023-12-17T12:08:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 --2- 172.21.15.27:0/2723563607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee64103020 0x7fee641098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 --2- 172.21.15.27:0/2723563607 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee64101490 0x7fee641018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 --2- 172.21.15.27:0/2723563607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64102680 0x7fee64102ae0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 -- 172.21.15.27:0/2723563607 >> 172.21.15.27:0/2723563607 conn(0x7fee640fca00 msgr2=0x7fee640fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 -- 172.21.15.27:0/2723563607 shutdown_connections 2023-12-17T12:08:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.535+0000 7fee69fff700 1 -- 172.21.15.27:0/2723563607 wait complete. 2023-12-17T12:08:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.536+0000 7fee69fff700 1 Processor -- start 2023-12-17T12:08:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.536+0000 7fee69fff700 1 -- start start 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.536+0000 7fee69fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64101490 0x7fee64199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee69fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee64102680 0x7fee64199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee69fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee64103020 0x7fee641a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee69fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641a1080 con 0x7fee64101490 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee69fff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641a11c0 con 0x7fee64103020 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee69fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641a1350 con 0x7fee64102680 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64101490 0x7fee64199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64101490 0x7fee64199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36584/0 (socket says 172.21.15.27:36584) 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee637fe700 1 -- 172.21.15.27:0/445446840 learned_addr learned my addr 172.21.15.27:0/445446840 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee62ffd700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee64102680 0x7fee64199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee63fff700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee64103020 0x7fee641a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee637fe700 1 -- 172.21.15.27:0/445446840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee64102680 msgr2=0x7fee64199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee637fe700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee64102680 0x7fee64199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee637fe700 1 -- 172.21.15.27:0/445446840 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee64103020 msgr2=0x7fee641a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee637fe700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee64103020 0x7fee641a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.537+0000 7fee637fe700 1 -- 172.21.15.27:0/445446840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580095f0 con 0x7fee64101490 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.538+0000 7fee637fe700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64101490 0x7fee64199550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e970 tx=0x7fee5400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.538+0000 7fee60ff9700 1 -- 172.21.15.27:0/445446840 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54009790 con 0x7fee64101490 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.538+0000 7fee60ff9700 1 -- 172.21.15.27:0/445446840 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee5400c7e0 con 0x7fee64101490 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.538+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee641a1630 con 0x7fee64101490 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.538+0000 7fee60ff9700 1 -- 172.21.15.27:0/445446840 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54003bd0 con 0x7fee64101490 2023-12-17T12:08:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.538+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee641a1b40 con 0x7fee64101490 2023-12-17T12:08:45.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.539+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee64068680 con 0x7fee64101490 2023-12-17T12:08:45.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.541+0000 7fee60ff9700 1 -- 172.21.15.27:0/445446840 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fee5400c950 con 0x7fee64101490 2023-12-17T12:08:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.543+0000 7fee60ff9700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee4c0a1480 0x7fee4c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.543+0000 7fee60ff9700 1 -- 172.21.15.27:0/445446840 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fee54014070 con 0x7fee64101490 2023-12-17T12:08:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.543+0000 7fee62ffd700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee4c0a1480 0x7fee4c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:45.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.544+0000 7fee62ffd700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee4c0a1480 0x7fee4c0a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fee58000c00 tx=0x7fee58003680 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:45.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.546+0000 7fee60ff9700 1 -- 172.21.15.27:0/445446840 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fee540966d0 con 0x7fee64101490 2023-12-17T12:08:45.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.827+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fee6410cbb0 con 0x7fee64101490 2023-12-17T12:08:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.828+0000 7fee60ff9700 1 -- 172.21.15.27:0/445446840 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7fee54099cf0 con 0x7fee64101490 2023-12-17T12:08:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:45.829 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-17T12:08:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-17T12:08:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-17T12:08:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-17T12:08:45.829 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-17T12:08:45.829 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee4c0a1480 msgr2=0x7fee4c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee4c0a1480 0x7fee4c0a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fee58000c00 tx=0x7fee58003680 comp rx=0 tx=0).stop 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64101490 msgr2=0x7fee64199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64101490 0x7fee64199550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e970 tx=0x7fee5400ec80 comp rx=0 tx=0).stop 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 shutdown_connections 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee4c0a1480 0x7fee4c0a3940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee64103020 0x7fee641a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64101490 0x7fee64199550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 --2- 172.21.15.27:0/445446840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee64102680 0x7fee64199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.830+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 >> 172.21.15.27:0/445446840 conn(0x7fee640fca00 msgr2=0x7fee64073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:45.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.831+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 shutdown_connections 2023-12-17T12:08:45.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:45.831+0000 7fee69fff700 1 -- 172.21.15.27:0/445446840 wait complete. 2023-12-17T12:08:45.852 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-17T12:08:45.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-17T12:08:46.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:45 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1137177272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:46.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:45 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1238172983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:46.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:45 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1137177272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:46.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:45 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1238172983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/1117927749 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd1f41015a0 msgr2=0x7fd1f41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/1117927749 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd1f41015a0 0x7fd1f41019c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00099c0 tx=0x7fd1f0009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/1117927749 shutdown_connections 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/1117927749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1f4103130 0x7fd1f41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/1117927749 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd1f41015a0 0x7fd1f41019c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/1117927749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4102790 0x7fd1f4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/1117927749 >> 172.21.15.27:0/1117927749 conn(0x7fd1f40fcb10 msgr2=0x7fd1f40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/1117927749 shutdown_connections 2023-12-17T12:08:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.093+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/1117927749 wait complete. 2023-12-17T12:08:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.094+0000 7fd1fc7ff700 1 Processor -- start 2023-12-17T12:08:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fc7ff700 1 -- start start 2023-12-17T12:08:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fc7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f41015a0 0x7fd1f4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fc7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1f4102790 0x7fd1f4199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fc7ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd1f4103130 0x7fd1f41a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fc7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f41a1240 con 0x7fd1f4102790 2023-12-17T12:08:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fc7ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f41a1380 con 0x7fd1f4103130 2023-12-17T12:08:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fc7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f41a1510 con 0x7fd1f41015a0 2023-12-17T12:08:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1f9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1f4102790 0x7fd1f4199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fa59b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f41015a0 0x7fd1f4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fa59b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f41015a0 0x7fd1f4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:60278/0 (socket says 172.21.15.27:60278) 2023-12-17T12:08:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1f9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1f4102790 0x7fd1f4199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38766/0 (socket says 172.21.15.27:38766) 2023-12-17T12:08:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fa59b700 1 -- 172.21.15.27:0/2996681854 learned_addr learned my addr 172.21.15.27:0/2996681854 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.095+0000 7fd1fad9c700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd1f4103130 0x7fd1f41a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1fa59b700 1 -- 172.21.15.27:0/2996681854 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd1f4103130 msgr2=0x7fd1f41a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1fa59b700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd1f4103130 0x7fd1f41a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1fa59b700 1 -- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1f4102790 msgr2=0x7fd1f4199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1fa59b700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1f4102790 0x7fd1f4199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1fa59b700 1 -- 172.21.15.27:0/2996681854 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00095f0 con 0x7fd1f41015a0 2023-12-17T12:08:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1f9d9a700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1f4102790 0x7fd1f4199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1fa59b700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f41015a0 0x7fd1f4199760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0000c00 tx=0x7fd1f000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1eb7fe700 1 -- 172.21.15.27:0/2996681854 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f001d070 con 0x7fd1f41015a0 2023-12-17T12:08:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1eb7fe700 1 -- 172.21.15.27:0/2996681854 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1f0005050 con 0x7fd1f41015a0 2023-12-17T12:08:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1eb7fe700 1 -- 172.21.15.27:0/2996681854 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0004a70 con 0x7fd1f41015a0 2023-12-17T12:08:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.096+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f41a1790 con 0x7fd1f41015a0 2023-12-17T12:08:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.097+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f41a1b60 con 0x7fd1f41015a0 2023-12-17T12:08:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.098+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f4068310 con 0x7fd1f41015a0 2023-12-17T12:08:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.100+0000 7fd1eb7fe700 1 -- 172.21.15.27:0/2996681854 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd1f00051c0 con 0x7fd1f41015a0 2023-12-17T12:08:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.101+0000 7fd1eb7fe700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd1e00a1480 0x7fd1e00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.101+0000 7fd1eb7fe700 1 -- 172.21.15.27:0/2996681854 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1f00cd5b0 con 0x7fd1f41015a0 2023-12-17T12:08:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.101+0000 7fd1f9d9a700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd1e00a1480 0x7fd1e00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.102+0000 7fd1f9d9a700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd1e00a1480 0x7fd1e00a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4011490 tx=0x7fd1e4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:46.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.103+0000 7fd1eb7fe700 1 -- 172.21.15.27:0/2996681854 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd1f00970a0 con 0x7fd1f41015a0 2023-12-17T12:08:46.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:45 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1137177272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:46.162 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:45 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1238172983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:08:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:46.326+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd1f410ccd0 con 0x7fd1f41015a0 2023-12-17T12:08:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.154+0000 7fd1eb7fe700 1 -- 172.21.15.27:0/2996681854 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd1f009a6c0 con 0x7fd1f41015a0 2023-12-17T12:08:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd1e00a1480 msgr2=0x7fd1e00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd1e00a1480 0x7fd1e00a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4011490 tx=0x7fd1e4009260 comp rx=0 tx=0).stop 2023-12-17T12:08:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f41015a0 msgr2=0x7fd1f4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f41015a0 0x7fd1f4199760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0000c00 tx=0x7fd1f000f690 comp rx=0 tx=0).stop 2023-12-17T12:08:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 shutdown_connections 2023-12-17T12:08:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd1e00a1480 0x7fd1e00a3940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f41015a0 0x7fd1f4199760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd1f4103130 0x7fd1f41a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 --2- 172.21.15.27:0/2996681854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1f4102790 0x7fd1f4199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.159+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 >> 172.21.15.27:0/2996681854 conn(0x7fd1f40fcb10 msgr2=0x7fd1f4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.160+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 shutdown_connections 2023-12-17T12:08:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.160+0000 7fd1fc7ff700 1 -- 172.21.15.27:0/2996681854 wait complete. 2023-12-17T12:08:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-17T12:08:47.181 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-17T12:08:47.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:47.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/445446840' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-17T12:08:47.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:47 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2996681854' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-17T12:08:47.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.401+0000 7f6cb3dff700 1 -- 172.21.15.27:0/3033411746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac1007b0 msgr2=0x7f6cac100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:47.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.401+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/3033411746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac1007b0 0x7f6cac100c30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0099c0 tx=0x7f6c9c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.402+0000 7f6cb3dff700 1 -- 172.21.15.27:0/3033411746 shutdown_connections 2023-12-17T12:08:47.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.402+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/3033411746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cac0ffe50 0x7f6cac100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.402+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/3033411746 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6cac101170 0x7f6cac1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.402+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/3033411746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac1007b0 0x7f6cac100c30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.402+0000 7f6cb3dff700 1 -- 172.21.15.27:0/3033411746 >> 172.21.15.27:0/3033411746 conn(0x7f6cac0fb9f0 msgr2=0x7f6cac0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:47.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.402+0000 7f6cb3dff700 1 -- 172.21.15.27:0/3033411746 shutdown_connections 2023-12-17T12:08:47.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.402+0000 7f6cb3dff700 1 -- 172.21.15.27:0/3033411746 wait complete. 2023-12-17T12:08:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb3dff700 1 Processor -- start 2023-12-17T12:08:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb3dff700 1 -- start start 2023-12-17T12:08:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb3dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac0ffe50 0x7f6cac1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb3dff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6cac1007b0 0x7f6cac199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb3dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cac101170 0x7f6cac1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb3dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac1a10d0 con 0x7f6cac0ffe50 2023-12-17T12:08:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb3dff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac1a1210 con 0x7f6cac1007b0 2023-12-17T12:08:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb3dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac1a13a0 con 0x7f6cac101170 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb1b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac0ffe50 0x7f6cac1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb1b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac0ffe50 0x7f6cac1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38786/0 (socket says 172.21.15.27:38786) 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb239c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cac101170 0x7f6cac1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.403+0000 7f6cb1b9b700 1 -- 172.21.15.27:0/2688097734 learned_addr learned my addr 172.21.15.27:0/2688097734 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.404+0000 7f6cb139a700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6cac1007b0 0x7f6cac199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.404+0000 7f6cb1b9b700 1 -- 172.21.15.27:0/2688097734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cac101170 msgr2=0x7f6cac1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.404+0000 7f6cb1b9b700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cac101170 0x7f6cac1a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.404+0000 7f6cb1b9b700 1 -- 172.21.15.27:0/2688097734 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6cac1007b0 msgr2=0x7f6cac199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.404+0000 7f6cb1b9b700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6cac1007b0 0x7f6cac199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.404+0000 7f6cb1b9b700 1 -- 172.21.15.27:0/2688097734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0095f0 con 0x7f6cac0ffe50 2023-12-17T12:08:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.404+0000 7f6cb239c700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cac101170 0x7f6cac1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.404+0000 7f6cb1b9b700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac0ffe50 0x7f6cac1995a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800e9a0 tx=0x7f6ca800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.405+0000 7f6ca2ffd700 1 -- 172.21.15.27:0/2688097734 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8009790 con 0x7f6cac0ffe50 2023-12-17T12:08:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.405+0000 7f6ca2ffd700 1 -- 172.21.15.27:0/2688097734 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ca8010430 con 0x7f6cac0ffe50 2023-12-17T12:08:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.405+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac1a1680 con 0x7f6cac0ffe50 2023-12-17T12:08:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.405+0000 7f6ca2ffd700 1 -- 172.21.15.27:0/2688097734 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800cea0 con 0x7f6cac0ffe50 2023-12-17T12:08:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.405+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac1a1b90 con 0x7f6cac0ffe50 2023-12-17T12:08:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.406+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac10aeb0 con 0x7f6cac0ffe50 2023-12-17T12:08:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.408+0000 7f6ca2ffd700 1 -- 172.21.15.27:0/2688097734 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6ca80105a0 con 0x7f6cac0ffe50 2023-12-17T12:08:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/445446840' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-17T12:08:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:47 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2996681854' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-17T12:08:47.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.411+0000 7f6ca2ffd700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6c980a1480 0x7f6c980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:47.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.411+0000 7f6ca2ffd700 1 -- 172.21.15.27:0/2688097734 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f6ca8014070 con 0x7f6cac0ffe50 2023-12-17T12:08:47.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.411+0000 7f6cb139a700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6c980a1480 0x7f6c980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.412+0000 7f6cb139a700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6c980a1480 0x7f6c980a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c006010 tx=0x7f6c9c003680 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:47.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.414+0000 7f6ca2ffd700 1 -- 172.21.15.27:0/2688097734 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6ca8096700 con 0x7f6cac0ffe50 2023-12-17T12:08:47.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:47.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/445446840' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-17T12:08:47.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:47.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:47 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2996681854' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-17T12:08:47.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:47.634+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6cac0682e0 con 0x7f6cac0ffe50 2023-12-17T12:08:48.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.374+0000 7f6ca2ffd700 1 -- 172.21.15.27:0/2688097734 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6ca8099d20 con 0x7f6cac0ffe50 2023-12-17T12:08:48.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6c980a1480 msgr2=0x7f6c980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6c980a1480 0x7f6c980a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c006010 tx=0x7f6c9c003680 comp rx=0 tx=0).stop 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac0ffe50 msgr2=0x7f6cac1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac0ffe50 0x7f6cac1995a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800e9a0 tx=0x7f6ca800ecb0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 shutdown_connections 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6c980a1480 0x7f6c980a3940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cac101170 0x7f6cac1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cac0ffe50 0x7f6cac1995a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 --2- 172.21.15.27:0/2688097734 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6cac1007b0 0x7f6cac199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.378+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 >> 172.21.15.27:0/2688097734 conn(0x7f6cac0fb9f0 msgr2=0x7f6cac0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:48.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.379+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 shutdown_connections 2023-12-17T12:08:48.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.379+0000 7f6cb3dff700 1 -- 172.21.15.27:0/2688097734 wait complete. 2023-12-17T12:08:48.385 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-17T12:08:48.419 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-17T12:08:48.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 -- 172.21.15.27:0/2758623436 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 msgr2=0x7fd5800744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:48.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 --2- 172.21.15.27:0/2758623436 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 0x7fd5800744b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009990 tx=0x7fd56c009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 -- 172.21.15.27:0/2758623436 shutdown_connections 2023-12-17T12:08:48.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 --2- 172.21.15.27:0/2758623436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58010db70 0x7fd58010ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 --2- 172.21.15.27:0/2758623436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd580075610 0x7fd580073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 --2- 172.21.15.27:0/2758623436 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 0x7fd5800744b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 -- 172.21.15.27:0/2758623436 >> 172.21.15.27:0/2758623436 conn(0x7fd5800fcb10 msgr2=0x7fd5800fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:48.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 -- 172.21.15.27:0/2758623436 shutdown_connections 2023-12-17T12:08:48.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.635+0000 7fd5871ff700 1 -- 172.21.15.27:0/2758623436 wait complete. 2023-12-17T12:08:48.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.636+0000 7fd5871ff700 1 Processor -- start 2023-12-17T12:08:48.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd5871ff700 1 -- start start 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd5871ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 0x7fd58019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd5871ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd580075610 0x7fd58019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd5871ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58010db70 0x7fd5801a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd5871ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5801a55d0 con 0x7fd580075610 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd5871ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5801a5710 con 0x7fd580074030 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd5871ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5801a58a0 con 0x7fd58010db70 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd584f9b700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 0x7fd58019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd584f9b700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 0x7fd58019daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:49208/0 (socket says 172.21.15.27:49208) 2023-12-17T12:08:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd584f9b700 1 -- 172.21.15.27:0/1277914087 learned_addr learned my addr 172.21.15.27:0/1277914087 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd57ffff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd580075610 0x7fd58019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.637+0000 7fd58579c700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58010db70 0x7fd5801a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.638+0000 7fd584f9b700 1 -- 172.21.15.27:0/1277914087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58010db70 msgr2=0x7fd5801a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.638+0000 7fd584f9b700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58010db70 0x7fd5801a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.638+0000 7fd584f9b700 1 -- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd580075610 msgr2=0x7fd58019e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.638+0000 7fd584f9b700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd580075610 0x7fd58019e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.638+0000 7fd584f9b700 1 -- 172.21.15.27:0/1277914087 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c0095f0 con 0x7fd580074030 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.638+0000 7fd584f9b700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 0x7fd58019daf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd57400e9a0 tx=0x7fd57400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.638+0000 7fd57dffb700 1 -- 172.21.15.27:0/1277914087 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574009790 con 0x7fd580074030 2023-12-17T12:08:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.639+0000 7fd57dffb700 1 -- 172.21.15.27:0/1277914087 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd574010430 con 0x7fd580074030 2023-12-17T12:08:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.639+0000 7fd57dffb700 1 -- 172.21.15.27:0/1277914087 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400cea0 con 0x7fd580074030 2023-12-17T12:08:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.639+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5801a5b80 con 0x7fd580074030 2023-12-17T12:08:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.639+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5801a6090 con 0x7fd580074030 2023-12-17T12:08:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.640+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd580103dd0 con 0x7fd580074030 2023-12-17T12:08:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.642+0000 7fd57dffb700 1 -- 172.21.15.27:0/1277914087 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd5740105a0 con 0x7fd580074030 2023-12-17T12:08:48.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.643+0000 7fd57dffb700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd5700a1500 0x7fd5700a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:48.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.644+0000 7fd57dffb700 1 -- 172.21.15.27:0/1277914087 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fd574014070 con 0x7fd580074030 2023-12-17T12:08:48.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.644+0000 7fd57ffff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd5700a1500 0x7fd5700a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:48.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.644+0000 7fd57ffff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd5700a1500 0x7fd5700a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005910 tx=0x7fd56c00b610 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:48.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.646+0000 7fd57dffb700 1 -- 172.21.15.27:0/1277914087 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd574096410 con 0x7fd580074030 2023-12-17T12:08:48.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:48 smithi027 ceph-mon[36871]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-17T12:08:48.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:48.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:48.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:48.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:48 smithi027 ceph-mon[36871]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-17T12:08:48.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:48 smithi027 ceph-mon[36871]: osdmap e79: 12 total, 12 up, 12 in 2023-12-17T12:08:48.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:48 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:48 smithi141 ceph-mon[41225]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-17T12:08:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:48 smithi141 ceph-mon[41225]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-17T12:08:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:48 smithi141 ceph-mon[41225]: osdmap e79: 12 total, 12 up, 12 in 2023-12-17T12:08:48.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:48 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:48 smithi192 ceph-mon[41899]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-17T12:08:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:48 smithi192 ceph-mon[41899]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-17T12:08:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:48 smithi192 ceph-mon[41899]: osdmap e79: 12 total, 12 up, 12 in 2023-12-17T12:08:48.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:48 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:08:48.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:48.914+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd580068310 con 0x7fd580074030 2023-12-17T12:08:49.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:49.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2688097734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-17T12:08:49.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2688097734' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-17T12:08:49.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: osdmap e80: 12 total, 12 up, 12 in 2023-12-17T12:08:49.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1277914087' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: osdmap e81: 12 total, 12 up, 12 in 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:49.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:49 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:49.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:49.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2688097734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-17T12:08:49.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2688097734' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-17T12:08:49.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: osdmap e80: 12 total, 12 up, 12 in 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1277914087' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: osdmap e81: 12 total, 12 up, 12 in 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:49.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:49 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:49.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:49.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2688097734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-17T12:08:49.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2688097734' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-17T12:08:49.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: osdmap e80: 12 total, 12 up, 12 in 2023-12-17T12:08:49.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1277914087' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-17T12:08:49.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-17T12:08:49.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: osdmap e81: 12 total, 12 up, 12 in 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:08:49.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:49 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:08:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.881+0000 7fd57dffb700 1 -- 172.21.15.27:0/1277914087 <== mon.2 v2:172.21.15.141:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd574099a30 con 0x7fd580074030 2023-12-17T12:08:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.883+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd5700a1500 msgr2=0x7fd5700a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.883+0000 7fd5871ff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd5700a1500 0x7fd5700a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005910 tx=0x7fd56c00b610 comp rx=0 tx=0).stop 2023-12-17T12:08:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.883+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 msgr2=0x7fd58019daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.883+0000 7fd5871ff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 0x7fd58019daf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd57400e9a0 tx=0x7fd57400ecb0 comp rx=0 tx=0).stop 2023-12-17T12:08:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.884+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 shutdown_connections 2023-12-17T12:08:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.884+0000 7fd5871ff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd5700a1500 0x7fd5700a39c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.884+0000 7fd5871ff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58010db70 0x7fd5801a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.884+0000 7fd5871ff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd580074030 0x7fd58019daf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.884+0000 7fd5871ff700 1 --2- 172.21.15.27:0/1277914087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd580075610 0x7fd58019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.884+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 >> 172.21.15.27:0/1277914087 conn(0x7fd5800fcb10 msgr2=0x7fd580100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.884+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 shutdown_connections 2023-12-17T12:08:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:49.885+0000 7fd5871ff700 1 -- 172.21.15.27:0/1277914087 wait complete. 2023-12-17T12:08:49.889 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2023-12-17T12:08:49.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:08:50.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 -- 172.21.15.27:0/2976691119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b101014a0 msgr2=0x7f6b101018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 --2- 172.21.15.27:0/2976691119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b101014a0 0x7f6b101018c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0099c0 tx=0x7f6b0c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 -- 172.21.15.27:0/2976691119 shutdown_connections 2023-12-17T12:08:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 --2- 172.21.15.27:0/2976691119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b10103030 0x7f6b101098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 --2- 172.21.15.27:0/2976691119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b101014a0 0x7f6b101018c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 --2- 172.21.15.27:0/2976691119 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6b10102690 0x7f6b10102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 -- 172.21.15.27:0/2976691119 >> 172.21.15.27:0/2976691119 conn(0x7f6b100fca10 msgr2=0x7f6b100fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 -- 172.21.15.27:0/2976691119 shutdown_connections 2023-12-17T12:08:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.117+0000 7f6b175ff700 1 -- 172.21.15.27:0/2976691119 wait complete. 2023-12-17T12:08:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.118+0000 7f6b175ff700 1 Processor -- start 2023-12-17T12:08:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.118+0000 7f6b175ff700 1 -- start start 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b175ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b101014a0 0x7f6b101973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b175ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6b10102690 0x7f6b101978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b175ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b10103030 0x7f6b1019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b175ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1019eed0 con 0x7f6b10103030 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b175ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1019f010 con 0x7f6b10102690 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b175ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1019f1a0 con 0x7f6b101014a0 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b15b9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b10103030 0x7f6b1019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b1539b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b101014a0 0x7f6b101973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b1539b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b101014a0 0x7f6b101973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:60326/0 (socket says 172.21.15.27:60326) 2023-12-17T12:08:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b1539b700 1 -- 172.21.15.27:0/3823107215 learned_addr learned my addr 172.21.15.27:0/3823107215 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b15b9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b10103030 0x7f6b1019e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38822/0 (socket says 172.21.15.27:38822) 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.119+0000 7f6b14b9a700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6b10102690 0x7f6b101978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b1539b700 1 -- 172.21.15.27:0/3823107215 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6b10102690 msgr2=0x7f6b101978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b1539b700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6b10102690 0x7f6b101978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b1539b700 1 -- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b10103030 msgr2=0x7f6b1019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b1539b700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b10103030 0x7f6b1019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b1539b700 1 -- 172.21.15.27:0/3823107215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0095f0 con 0x7f6b101014a0 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b15b9c700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b10103030 0x7f6b1019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b1539b700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b101014a0 0x7f6b101973a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009f50 tx=0x7f6b0c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b067fc700 1 -- 172.21.15.27:0/3823107215 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c01d070 con 0x7f6b101014a0 2023-12-17T12:08:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b067fc700 1 -- 172.21.15.27:0/3823107215 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b0c00bda0 con 0x7f6b101014a0 2023-12-17T12:08:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1019f420 con 0x7f6b101014a0 2023-12-17T12:08:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b067fc700 1 -- 172.21.15.27:0/3823107215 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c022d30 con 0x7f6b101014a0 2023-12-17T12:08:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.120+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1019f840 con 0x7f6b101014a0 2023-12-17T12:08:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.121+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b10108960 con 0x7f6b101014a0 2023-12-17T12:08:50.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.123+0000 7f6b067fc700 1 -- 172.21.15.27:0/3823107215 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6b0c022500 con 0x7f6b101014a0 2023-12-17T12:08:50.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.125+0000 7f6b067fc700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6afc0a1480 0x7f6afc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:50.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.125+0000 7f6b067fc700 1 -- 172.21.15.27:0/3823107215 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6b0c0ce650 con 0x7f6b101014a0 2023-12-17T12:08:50.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.125+0000 7f6b14b9a700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6afc0a1480 0x7f6afc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:50.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.126+0000 7f6b14b9a700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6afc0a1480 0x7f6afc0a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b00005950 tx=0x7f6b0000b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:50.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.127+0000 7f6b067fc700 1 -- 172.21.15.27:0/3823107215 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6b0c097df0 con 0x7f6b101014a0 2023-12-17T12:08:50.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.343+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b1010ccc0 con 0x7f6b101014a0 2023-12-17T12:08:50.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.345+0000 7f6b067fc700 1 -- 172.21.15.27:0/3823107215 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22130 (secure 0 0 0) 0x7f6b0c09b410 con 0x7f6b101014a0 2023-12-17T12:08:50.347 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:50.347 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:08:50.161605+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6afc0a1480 msgr2=0x7f6afc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6afc0a1480 0x7f6afc0a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b00005950 tx=0x7f6b0000b410 comp rx=0 tx=0).stop 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b101014a0 msgr2=0x7f6b101973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b101014a0 0x7f6b101973a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009f50 tx=0x7f6b0c005b00 comp rx=0 tx=0).stop 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 shutdown_connections 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f6afc0a1480 0x7f6afc0a3940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b101014a0 0x7f6b101973a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b10103030 0x7f6b1019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 --2- 172.21.15.27:0/3823107215 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f6b10102690 0x7f6b101978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 >> 172.21.15.27:0/3823107215 conn(0x7f6b100fca10 msgr2=0x7f6b10105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:50.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 shutdown_connections 2023-12-17T12:08:50.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.347+0000 7f6b175ff700 1 -- 172.21.15.27:0/3823107215 wait complete. 2023-12-17T12:08:50.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:08:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3429990832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c4103030 msgr2=0x7fd3c41098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3429990832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c4103030 0x7fd3c41098e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00b0d0 tx=0x7fd3bc00b3e0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3429990832 shutdown_connections 2023-12-17T12:08:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3429990832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c41014a0 0x7fd3c41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3429990832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c4103030 0x7fd3c41098e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3429990832 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd3c4102690 0x7fd3c4102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3429990832 >> 172.21.15.27:0/3429990832 conn(0x7fd3c40fca10 msgr2=0x7fd3c40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3429990832 shutdown_connections 2023-12-17T12:08:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.575+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3429990832 wait complete. 2023-12-17T12:08:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.576+0000 7fd3c95ff700 1 Processor -- start 2023-12-17T12:08:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c95ff700 1 -- start start 2023-12-17T12:08:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c95ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c41014a0 0x7fd3c41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c95ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c4102690 0x7fd3c4199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c95ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd3c4103030 0x7fd3c41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c95ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c41a11c0 con 0x7fd3c41014a0 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c95ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c41a1300 con 0x7fd3c4103030 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c95ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c41a1490 con 0x7fd3c4102690 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c41014a0 0x7fd3c41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c41014a0 0x7fd3c41996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38850/0 (socket says 172.21.15.27:38850) 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.577+0000 7fd3c2ffd700 1 -- 172.21.15.27:0/3687119473 learned_addr learned my addr 172.21.15.27:0/3687119473 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c37fe700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd3c4103030 0x7fd3c41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c27fc700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c4102690 0x7fd3c4199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c2ffd700 1 -- 172.21.15.27:0/3687119473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c4102690 msgr2=0x7fd3c4199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c2ffd700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c4102690 0x7fd3c4199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c2ffd700 1 -- 172.21.15.27:0/3687119473 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd3c4103030 msgr2=0x7fd3c41a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c2ffd700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd3c4103030 0x7fd3c41a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c2ffd700 1 -- 172.21.15.27:0/3687119473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc009d00 con 0x7fd3c41014a0 2023-12-17T12:08:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c2ffd700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c41014a0 0x7fd3c41996e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400d980 tx=0x7fd3b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3abfff700 1 -- 172.21.15.27:0/3687119473 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4004020 con 0x7fd3c41014a0 2023-12-17T12:08:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3abfff700 1 -- 172.21.15.27:0/3687119473 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3b400f690 con 0x7fd3c41014a0 2023-12-17T12:08:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c41a1770 con 0x7fd3c41014a0 2023-12-17T12:08:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.578+0000 7fd3abfff700 1 -- 172.21.15.27:0/3687119473 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40039e0 con 0x7fd3c41014a0 2023-12-17T12:08:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.579+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c41a1c80 con 0x7fd3c41014a0 2023-12-17T12:08:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.580+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c4068230 con 0x7fd3c41014a0 2023-12-17T12:08:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.582+0000 7fd3abfff700 1 -- 172.21.15.27:0/3687119473 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd3b400f800 con 0x7fd3c41014a0 2023-12-17T12:08:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.584+0000 7fd3abfff700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd3ac0a1480 0x7fd3ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.584+0000 7fd3abfff700 1 -- 172.21.15.27:0/3687119473 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3b40cc810 con 0x7fd3c41014a0 2023-12-17T12:08:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.584+0000 7fd3c27fc700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd3ac0a1480 0x7fd3ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.585+0000 7fd3c27fc700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd3ac0a1480 0x7fd3ac0a3940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005950 tx=0x7fd3b800b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:50.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.589+0000 7fd3abfff700 1 -- 172.21.15.27:0/3687119473 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd3b4095f20 con 0x7fd3c41014a0 2023-12-17T12:08:50.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:50 smithi027 ceph-mon[36871]: pgmap v263: 97 pgs: 11 creating+peering, 80 unknown, 6 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:50.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:50 smithi027 ceph-mon[36871]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-17T12:08:50.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:50.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:50 smithi027 ceph-mon[36871]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-17T12:08:50.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:50 smithi027 ceph-mon[36871]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-17T12:08:50.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:50 smithi027 ceph-mon[36871]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-17T12:08:50.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:50 smithi027 ceph-mon[36871]: osdmap e82: 12 total, 12 up, 12 in 2023-12-17T12:08:50.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:50 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3823107215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:50 smithi141 ceph-mon[41225]: pgmap v263: 97 pgs: 11 creating+peering, 80 unknown, 6 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:50 smithi141 ceph-mon[41225]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-17T12:08:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:50 smithi141 ceph-mon[41225]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-17T12:08:50.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:50 smithi141 ceph-mon[41225]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-17T12:08:50.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:50 smithi141 ceph-mon[41225]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-17T12:08:50.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:50 smithi141 ceph-mon[41225]: osdmap e82: 12 total, 12 up, 12 in 2023-12-17T12:08:50.765 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:50 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3823107215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:50.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.808+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3c410cbb0 con 0x7fd3c41014a0 2023-12-17T12:08:50.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:50 smithi192 ceph-mon[41899]: pgmap v263: 97 pgs: 11 creating+peering, 80 unknown, 6 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-17T12:08:50.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:50 smithi192 ceph-mon[41899]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-17T12:08:50.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:08:50.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:50 smithi192 ceph-mon[41899]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-17T12:08:50.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:50 smithi192 ceph-mon[41899]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-17T12:08:50.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:50 smithi192 ceph-mon[41899]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-17T12:08:50.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:50 smithi192 ceph-mon[41899]: osdmap e82: 12 total, 12 up, 12 in 2023-12-17T12:08:50.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:50 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3823107215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:50.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.810+0000 7fd3abfff700 1 -- 172.21.15.27:0/3687119473 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22130 (secure 0 0 0) 0x7fd3b4099540 con 0x7fd3c41014a0 2023-12-17T12:08:50.812 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:50.812 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:08:50.161605+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd3ac0a1480 msgr2=0x7fd3ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd3ac0a1480 0x7fd3ac0a3940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005950 tx=0x7fd3b800b410 comp rx=0 tx=0).stop 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c41014a0 msgr2=0x7fd3c41996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c41014a0 0x7fd3c41996e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400d980 tx=0x7fd3b400dc90 comp rx=0 tx=0).stop 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 shutdown_connections 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd3ac0a1480 0x7fd3ac0a3940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd3c4103030 0x7fd3c41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3c41014a0 0x7fd3c41996e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 --2- 172.21.15.27:0/3687119473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c4102690 0x7fd3c4199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 >> 172.21.15.27:0/3687119473 conn(0x7fd3c40fca10 msgr2=0x7fd3c4073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 shutdown_connections 2023-12-17T12:08:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:50.812+0000 7fd3c95ff700 1 -- 172.21.15.27:0/3687119473 wait complete. 2023-12-17T12:08:50.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 -- 172.21.15.27:0/1976719379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 msgr2=0x7f412410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 --2- 172.21.15.27:0/1976719379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 0x7f412410b8e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f411c00b0a0 tx=0x7f411c00b3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 -- 172.21.15.27:0/1976719379 shutdown_connections 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 --2- 172.21.15.27:0/1976719379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41240755b0 0x7f4124073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 --2- 172.21.15.27:0/1976719379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 0x7f412410b8e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 --2- 172.21.15.27:0/1976719379 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4124073fd0 0x7f4124074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 -- 172.21.15.27:0/1976719379 >> 172.21.15.27:0/1976719379 conn(0x7f41240fcb10 msgr2=0x7f41240fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 -- 172.21.15.27:0/1976719379 shutdown_connections 2023-12-17T12:08:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.043+0000 7f412c1ff700 1 -- 172.21.15.27:0/1976719379 wait complete. 2023-12-17T12:08:51.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.044+0000 7f412c1ff700 1 Processor -- start 2023-12-17T12:08:51.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.044+0000 7f412c1ff700 1 -- start start 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412c1ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4124073fd0 0x7f41241996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412c1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41240755b0 0x7f4124199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412c1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 0x7f41241a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412c1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41241a11f0 con 0x7f4124109470 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412c1ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41241a1330 con 0x7f4124073fd0 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412c1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41241a14c0 con 0x7f41240755b0 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412a79c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 0x7f41241a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412a79c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 0x7f41241a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38860/0 (socket says 172.21.15.27:38860) 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412a79c700 1 -- 172.21.15.27:0/4277025139 learned_addr learned my addr 172.21.15.27:0/4277025139 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f412979a700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41240755b0 0x7f4124199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.045+0000 7f4129f9b700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4124073fd0 0x7f41241996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f412a79c700 1 -- 172.21.15.27:0/4277025139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41240755b0 msgr2=0x7f4124199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f412a79c700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41240755b0 0x7f4124199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f412a79c700 1 -- 172.21.15.27:0/4277025139 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4124073fd0 msgr2=0x7f41241996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f412a79c700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4124073fd0 0x7f41241996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f412a79c700 1 -- 172.21.15.27:0/4277025139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009d00 con 0x7f4124109470 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f412a79c700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 0x7f41241a0c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f411c00b780 tx=0x7f411c00bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f411affd700 1 -- 172.21.15.27:0/4277025139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c00e040 con 0x7f4124109470 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f411affd700 1 -- 172.21.15.27:0/4277025139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f411c009480 con 0x7f4124109470 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41241a1740 con 0x7f4124109470 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f411affd700 1 -- 172.21.15.27:0/4277025139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0052c0 con 0x7f4124109470 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.046+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41241a1b60 con 0x7f4124109470 2023-12-17T12:08:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.047+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4124108280 con 0x7f4124109470 2023-12-17T12:08:51.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.049+0000 7f411affd700 1 -- 172.21.15.27:0/4277025139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f411c01d9a0 con 0x7f4124109470 2023-12-17T12:08:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.051+0000 7f411affd700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41100a1480 0x7f41100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.051+0000 7f411affd700 1 -- 172.21.15.27:0/4277025139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f411c0cdfb0 con 0x7f4124109470 2023-12-17T12:08:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.051+0000 7f4129f9b700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41100a1480 0x7f41100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.052+0000 7f4129f9b700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41100a1480 0x7f41100a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4120005950 tx=0x7f412000b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:51.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.053+0000 7f411affd700 1 -- 172.21.15.27:0/4277025139 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f411c0977f0 con 0x7f4124109470 2023-12-17T12:08:51.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.316+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4124068310 con 0x7f4124109470 2023-12-17T12:08:51.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:51 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3687119473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:51.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:51 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] up:active 2023-12-17T12:08:51.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:51 smithi027 ceph-mon[36871]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:51.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:51 smithi027 ceph-mon[36871]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:51.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:51 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4277025139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-17T12:08:51.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:51 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3687119473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:51.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:51 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] up:active 2023-12-17T12:08:51.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:51 smithi141 ceph-mon[41225]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:51.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:51 smithi141 ceph-mon[41225]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:51.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:51 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4277025139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-17T12:08:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:51 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3687119473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:51 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] up:active 2023-12-17T12:08:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:51 smithi192 ceph-mon[41899]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:51 smithi192 ceph-mon[41899]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:51.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:51 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4277025139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-17T12:08:51.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.888+0000 7f411affd700 1 -- 172.21.15.27:0/4277025139 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f411c09ae10 con 0x7f4124109470 2023-12-17T12:08:51.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41100a1480 msgr2=0x7f41100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41100a1480 0x7f41100a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4120005950 tx=0x7f412000b410 comp rx=0 tx=0).stop 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 msgr2=0x7f41241a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 0x7f41241a0c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f411c00b780 tx=0x7f411c00bcc0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 shutdown_connections 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41100a1480 0x7f41100a3940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124109470 0x7f41241a0c20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4124073fd0 0x7f41241996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 --2- 172.21.15.27:0/4277025139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41240755b0 0x7f4124199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 >> 172.21.15.27:0/4277025139 conn(0x7f41240fcb10 msgr2=0x7f41241053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 shutdown_connections 2023-12-17T12:08:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:51.892+0000 7f412c1ff700 1 -- 172.21.15.27:0/4277025139 wait complete. 2023-12-17T12:08:51.914 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-12-17T12:08:51.914 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 -- 172.21.15.27:0/4213022979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b8107320 msgr2=0x7f09b8109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 --2- 172.21.15.27:0/4213022979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b8107320 0x7f09b8109710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f09b000b0a0 tx=0x7f09b000b3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 -- 172.21.15.27:0/4213022979 shutdown_connections 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 --2- 172.21.15.27:0/4213022979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b81020c0 0x7f09b81044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 --2- 172.21.15.27:0/4213022979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b8107320 0x7f09b8109710 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 --2- 172.21.15.27:0/4213022979 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f09b81049f0 0x7f09b8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 -- 172.21.15.27:0/4213022979 >> 172.21.15.27:0/4213022979 conn(0x7f09b80fb9f0 msgr2=0x7f09b80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 -- 172.21.15.27:0/4213022979 shutdown_connections 2023-12-17T12:08:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.123+0000 7f09befff700 1 -- 172.21.15.27:0/4213022979 wait complete. 2023-12-17T12:08:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.124+0000 7f09befff700 1 Processor -- start 2023-12-17T12:08:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.124+0000 7f09befff700 1 -- start start 2023-12-17T12:08:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09befff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b81020c0 0x7f09b81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09befff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81049f0 0x7f09b8199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09befff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f09b8107320 0x7f09b81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09befff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b81a1220 con 0x7f09b81049f0 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09befff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b81a1360 con 0x7f09b8107320 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09befff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b81a14f0 con 0x7f09b81020c0 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09b7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81049f0 0x7f09b8199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09bcd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b81020c0 0x7f09b81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09b7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81049f0 0x7f09b8199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38878/0 (socket says 172.21.15.27:38878) 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09bcd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b81020c0 0x7f09b81996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:60384/0 (socket says 172.21.15.27:60384) 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09b7fff700 1 -- 172.21.15.27:0/848491922 learned_addr learned my addr 172.21.15.27:0/848491922 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.125+0000 7f09bd59c700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f09b8107320 0x7f09b81a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b7fff700 1 -- 172.21.15.27:0/848491922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b81020c0 msgr2=0x7f09b81996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b7fff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b81020c0 0x7f09b81996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b7fff700 1 -- 172.21.15.27:0/848491922 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f09b8107320 msgr2=0x7f09b81a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b7fff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f09b8107320 0x7f09b81a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b7fff700 1 -- 172.21.15.27:0/848491922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0009d00 con 0x7f09b81049f0 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b7fff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81049f0 0x7f09b8199c30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f09a400e9a0 tx=0x7f09a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b5ffb700 1 -- 172.21.15.27:0/848491922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4009790 con 0x7f09b81049f0 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b5ffb700 1 -- 172.21.15.27:0/848491922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09a4010430 con 0x7f09b81049f0 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b81a17d0 con 0x7f09b81049f0 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09b5ffb700 1 -- 172.21.15.27:0/848491922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a400cea0 con 0x7f09b81049f0 2023-12-17T12:08:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.126+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b81a1ce0 con 0x7f09b81049f0 2023-12-17T12:08:52.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.127+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b8108ea0 con 0x7f09b81049f0 2023-12-17T12:08:52.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.130+0000 7f09b5ffb700 1 -- 172.21.15.27:0/848491922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f09a40105a0 con 0x7f09b81049f0 2023-12-17T12:08:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.131+0000 7f09b5ffb700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09a80a1480 0x7f09a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.131+0000 7f09b5ffb700 1 -- 172.21.15.27:0/848491922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09a4014070 con 0x7f09b81049f0 2023-12-17T12:08:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.131+0000 7f09bcd9b700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09a80a1480 0x7f09a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.132+0000 7f09bcd9b700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09a80a1480 0x7f09a80a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005950 tx=0x7f09ac00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:52.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.135+0000 7f09b5ffb700 1 -- 172.21.15.27:0/848491922 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f09a40522f0 con 0x7f09b81049f0 2023-12-17T12:08:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:52.397+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f09b8108880 con 0x7f09b81049f0 2023-12-17T12:08:52.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:52 smithi027 ceph-mon[36871]: pgmap v266: 97 pgs: 25 creating+peering, 37 unknown, 35 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-17T12:08:52.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:52 smithi027 ceph-mon[36871]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-17T12:08:52.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:52 smithi027 ceph-mon[36871]: Cluster is now healthy 2023-12-17T12:08:52.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:52 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4277025139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-17T12:08:52.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:52 smithi027 ceph-mon[36871]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:52.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:52 smithi141 ceph-mon[41225]: pgmap v266: 97 pgs: 25 creating+peering, 37 unknown, 35 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-17T12:08:52.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:52 smithi141 ceph-mon[41225]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-17T12:08:52.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:52 smithi141 ceph-mon[41225]: Cluster is now healthy 2023-12-17T12:08:52.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:52 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4277025139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-17T12:08:52.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:52 smithi141 ceph-mon[41225]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:52.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:52 smithi192 ceph-mon[41899]: pgmap v266: 97 pgs: 25 creating+peering, 37 unknown, 35 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-17T12:08:52.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:52 smithi192 ceph-mon[41899]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-17T12:08:52.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:52 smithi192 ceph-mon[41899]: Cluster is now healthy 2023-12-17T12:08:52.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:52 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4277025139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-17T12:08:52.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:52 smithi192 ceph-mon[41899]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.397+0000 7f09b5ffb700 1 -- 172.21.15.27:0/848491922 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f09a4096520 con 0x7f09b81049f0 2023-12-17T12:08:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09a80a1480 msgr2=0x7f09a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09a80a1480 0x7f09a80a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005950 tx=0x7f09ac00b410 comp rx=0 tx=0).stop 2023-12-17T12:08:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81049f0 msgr2=0x7f09b8199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81049f0 0x7f09b8199c30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f09a400e9a0 tx=0x7f09a400ecb0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 shutdown_connections 2023-12-17T12:08:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09a80a1480 0x7f09a80a3940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b81020c0 0x7f09b81996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f09b8107320 0x7f09b81a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 --2- 172.21.15.27:0/848491922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81049f0 0x7f09b8199c30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 >> 172.21.15.27:0/848491922 conn(0x7f09b80fb9f0 msgr2=0x7f09b81018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 shutdown_connections 2023-12-17T12:08:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.401+0000 7f09befff700 1 -- 172.21.15.27:0/848491922 wait complete. 2023-12-17T12:08:53.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-12-17T12:08:53.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 -- 172.21.15.27:0/3711228192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84281015a0 msgr2=0x7f84281019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 --2- 172.21.15.27:0/3711228192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84281015a0 0x7f84281019c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84240099c0 tx=0x7f8424009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 -- 172.21.15.27:0/3711228192 shutdown_connections 2023-12-17T12:08:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 --2- 172.21.15.27:0/3711228192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8428103130 0x7f84281099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 --2- 172.21.15.27:0/3711228192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84281015a0 0x7f84281019c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 --2- 172.21.15.27:0/3711228192 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8428102790 0x7f8428102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 -- 172.21.15.27:0/3711228192 >> 172.21.15.27:0/3711228192 conn(0x7f84280fcb10 msgr2=0x7f84280fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 -- 172.21.15.27:0/3711228192 shutdown_connections 2023-12-17T12:08:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.644+0000 7f842f9ff700 1 -- 172.21.15.27:0/3711228192 wait complete. 2023-12-17T12:08:53.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.645+0000 7f842f9ff700 1 Processor -- start 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.645+0000 7f842f9ff700 1 -- start start 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.645+0000 7f842f9ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f84281015a0 0x7f84281995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.645+0000 7f842f9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8428102790 0x7f8428199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842f9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8428103130 0x7f84281a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842f9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a1120 con 0x7f8428103130 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842f9ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a1260 con 0x7f84281015a0 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842f9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a13f0 con 0x7f8428102790 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8428103130 0x7f84281a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8428103130 0x7f84281a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38900/0 (socket says 172.21.15.27:38900) 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 -- 172.21.15.27:0/1049334969 learned_addr learned my addr 172.21.15.27:0/1049334969 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842d79b700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f84281015a0 0x7f84281995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842cf9a700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8428102790 0x7f8428199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 -- 172.21.15.27:0/1049334969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8428102790 msgr2=0x7f8428199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8428102790 0x7f8428199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 -- 172.21.15.27:0/1049334969 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f84281015a0 msgr2=0x7f84281995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f84281015a0 0x7f84281995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 -- 172.21.15.27:0/1049334969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84240095f0 con 0x7f8428103130 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.646+0000 7f842df9c700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8428103130 0x7f84281a0b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8420007ee0 tx=0x7f842000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.647+0000 7f841e7fc700 1 -- 172.21.15.27:0/1049334969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420017070 con 0x7f8428103130 2023-12-17T12:08:53.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.647+0000 7f841e7fc700 1 -- 172.21.15.27:0/1049334969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f842000f040 con 0x7f8428103130 2023-12-17T12:08:53.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.647+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84281a16d0 con 0x7f8428103130 2023-12-17T12:08:53.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.647+0000 7f841e7fc700 1 -- 172.21.15.27:0/1049334969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420005710 con 0x7f8428103130 2023-12-17T12:08:53.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.647+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84281a1be0 con 0x7f8428103130 2023-12-17T12:08:53.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.648+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8428068310 con 0x7f8428103130 2023-12-17T12:08:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.650+0000 7f841e7fc700 1 -- 172.21.15.27:0/1049334969 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8420005870 con 0x7f8428103130 2023-12-17T12:08:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.651+0000 7f841e7fc700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f84140a1500 0x7f84140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.651+0000 7f841e7fc700 1 -- 172.21.15.27:0/1049334969 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84200cca60 con 0x7f8428103130 2023-12-17T12:08:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.652+0000 7f842d79b700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f84140a1500 0x7f84140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:53.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.652+0000 7f842d79b700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f84140a1500 0x7f84140a39c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8424006010 tx=0x7f8424000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:53.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.654+0000 7f841e7fc700 1 -- 172.21.15.27:0/1049334969 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8420096170 con 0x7f8428103130 2023-12-17T12:08:53.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:53 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/848491922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-17T12:08:53.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:53 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/848491922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-17T12:08:53.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:53 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/848491922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-17T12:08:53.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:53.913+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f842810cc70 con 0x7f8428103130 2023-12-17T12:08:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.406+0000 7f841e7fc700 1 -- 172.21.15.27:0/1049334969 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f8420099790 con 0x7f8428103130 2023-12-17T12:08:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f84140a1500 msgr2=0x7f84140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f84140a1500 0x7f84140a39c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8424006010 tx=0x7f8424000bc0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8428103130 msgr2=0x7f84281a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8428103130 0x7f84281a0b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8420007ee0 tx=0x7f842000d3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 shutdown_connections 2023-12-17T12:08:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f84140a1500 0x7f84140a39c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8428103130 0x7f84281a0b50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f84281015a0 0x7f84281995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 --2- 172.21.15.27:0/1049334969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8428102790 0x7f8428199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 >> 172.21.15.27:0/1049334969 conn(0x7f84280fcb10 msgr2=0x7f8428107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 shutdown_connections 2023-12-17T12:08:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.409+0000 7f842f9ff700 1 -- 172.21.15.27:0/1049334969 wait complete. 2023-12-17T12:08:54.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-12-17T12:08:54.514 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-b[77436]: 2023-12-17T12:08:54.245+0000 7f6f8dcb1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-17T12:08:54.514 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-mon[41225]: pgmap v267: 97 pgs: 22 creating+peering, 9 unknown, 66 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2023-12-17T12:08:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-mon[41225]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-17T12:08:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/848491922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-17T12:08:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-mon[41225]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-17T12:08:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-mon[41225]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-mon[41225]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-17T12:08:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-mon[41225]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-17T12:08:54.515 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:54 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1049334969' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-17T12:08:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/2729525168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84102790 msgr2=0x7f2a84102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/2729525168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84102790 0x7f2a84102bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0099c0 tx=0x7f2a6c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/2729525168 shutdown_connections 2023-12-17T12:08:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/2729525168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a84103130 0x7f2a841099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/2729525168 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a841015a0 0x7f2a841019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/2729525168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84102790 0x7f2a84102bf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/2729525168 >> 172.21.15.27:0/2729525168 conn(0x7f2a840fcb10 msgr2=0x7f2a840fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/2729525168 shutdown_connections 2023-12-17T12:08:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.646+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/2729525168 wait complete. 2023-12-17T12:08:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.647+0000 7f2a8a1ff700 1 Processor -- start 2023-12-17T12:08:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.647+0000 7f2a8a1ff700 1 -- start start 2023-12-17T12:08:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.647+0000 7f2a8a1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a841015a0 0x7f2a84199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.647+0000 7f2a8a1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a84102790 0x7f2a84199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.647+0000 7f2a8a1ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a84103130 0x7f2a841a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.647+0000 7f2a8a1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a11e0 con 0x7f2a841015a0 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.647+0000 7f2a8a1ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a1320 con 0x7f2a84103130 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.647+0000 7f2a8a1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a14b0 con 0x7f2a84102790 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a841015a0 0x7f2a84199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a841015a0 0x7f2a84199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38916/0 (socket says 172.21.15.27:38916) 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 -- 172.21.15.27:0/3735067277 learned_addr learned my addr 172.21.15.27:0/3735067277 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a83fff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a84103130 0x7f2a841a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a7bfff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a84102790 0x7f2a84199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 -- 172.21.15.27:0/3735067277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a84102790 msgr2=0x7f2a84199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a84102790 0x7f2a84199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 -- 172.21.15.27:0/3735067277 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a84103130 msgr2=0x7f2a841a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a84103130 0x7f2a841a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 -- 172.21.15.27:0/3735067277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c0095f0 con 0x7f2a841015a0 2023-12-17T12:08:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a837fe700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a841015a0 0x7f2a84199700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400d980 tx=0x7f2a7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.648+0000 7f2a817fa700 1 -- 172.21.15.27:0/3735067277 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74004020 con 0x7f2a841015a0 2023-12-17T12:08:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.649+0000 7f2a817fa700 1 -- 172.21.15.27:0/3735067277 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a7400f690 con 0x7f2a841015a0 2023-12-17T12:08:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.649+0000 7f2a817fa700 1 -- 172.21.15.27:0/3735067277 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740039e0 con 0x7f2a841015a0 2023-12-17T12:08:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.649+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a841a1790 con 0x7f2a841015a0 2023-12-17T12:08:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.649+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a841a1c50 con 0x7f2a841015a0 2023-12-17T12:08:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.650+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a84068310 con 0x7f2a841015a0 2023-12-17T12:08:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.652+0000 7f2a817fa700 1 -- 172.21.15.27:0/3735067277 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2a7400f800 con 0x7f2a841015a0 2023-12-17T12:08:54.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.653+0000 7f2a817fa700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a700a1480 0x7f2a700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.653+0000 7f2a817fa700 1 -- 172.21.15.27:0/3735067277 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2a740cc810 con 0x7f2a841015a0 2023-12-17T12:08:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.653+0000 7f2a7bfff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a700a1480 0x7f2a700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.654+0000 7f2a7bfff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a700a1480 0x7f2a700a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b580 tx=0x7f2a6c005e80 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:54.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.657+0000 7f2a817fa700 1 -- 172.21.15.27:0/3735067277 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2a74095f20 con 0x7f2a841015a0 2023-12-17T12:08:54.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:54 smithi027 ceph-mon[36871]: pgmap v267: 97 pgs: 22 creating+peering, 9 unknown, 66 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2023-12-17T12:08:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:54 smithi027 ceph-mon[36871]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-17T12:08:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:54 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/848491922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-17T12:08:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:54 smithi027 ceph-mon[36871]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-17T12:08:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:54 smithi027 ceph-mon[36871]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:54 smithi027 ceph-mon[36871]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-17T12:08:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:54 smithi027 ceph-mon[36871]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-17T12:08:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:54 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1049334969' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-17T12:08:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:54 smithi192 ceph-mon[41899]: pgmap v267: 97 pgs: 22 creating+peering, 9 unknown, 66 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2023-12-17T12:08:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:54 smithi192 ceph-mon[41899]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-17T12:08:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:54 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/848491922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-17T12:08:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:54 smithi192 ceph-mon[41899]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-17T12:08:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:54 smithi192 ceph-mon[41899]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-17T12:08:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:54 smithi192 ceph-mon[41899]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-17T12:08:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:54 smithi192 ceph-mon[41899]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-17T12:08:54.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:54 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1049334969' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-17T12:08:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:54.915+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f2a8410ccc0 con 0x7f2a841015a0 2023-12-17T12:08:55.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.416+0000 7f2a817fa700 1 -- 172.21.15.27:0/3735067277 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2a74099540 con 0x7f2a841015a0 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a700a1480 msgr2=0x7f2a700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a700a1480 0x7f2a700a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b580 tx=0x7f2a6c005e80 comp rx=0 tx=0).stop 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a841015a0 msgr2=0x7f2a84199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a841015a0 0x7f2a84199700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400d980 tx=0x7f2a7400dc90 comp rx=0 tx=0).stop 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 shutdown_connections 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2a700a1480 0x7f2a700a3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2a84103130 0x7f2a841a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a841015a0 0x7f2a84199700 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.419+0000 7f2a8a1ff700 1 --2- 172.21.15.27:0/3735067277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a84102790 0x7f2a84199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.420+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 >> 172.21.15.27:0/3735067277 conn(0x7f2a840fcb10 msgr2=0x7f2a84107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:55.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.420+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 shutdown_connections 2023-12-17T12:08:55.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.420+0000 7f2a8a1ff700 1 -- 172.21.15.27:0/3735067277 wait complete. 2023-12-17T12:08:55.442 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-12-17T12:08:55.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.653+0000 7f0c10dff700 1 -- 172.21.15.27:0/2704377573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 msgr2=0x7f0c0c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.653+0000 7f0c10dff700 1 --2- 172.21.15.27:0/2704377573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 0x7f0c0c1019c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0099c0 tx=0x7f0bfc009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.653+0000 7f0c10dff700 1 -- 172.21.15.27:0/2704377573 shutdown_connections 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.653+0000 7f0c10dff700 1 --2- 172.21.15.27:0/2704377573 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0c0c103130 0x7f0c0c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.653+0000 7f0c10dff700 1 --2- 172.21.15.27:0/2704377573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 0x7f0c0c1019c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.653+0000 7f0c10dff700 1 --2- 172.21.15.27:0/2704377573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c0c102790 0x7f0c0c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.653+0000 7f0c10dff700 1 -- 172.21.15.27:0/2704377573 >> 172.21.15.27:0/2704377573 conn(0x7f0c0c0fcb10 msgr2=0x7f0c0c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.653+0000 7f0c10dff700 1 -- 172.21.15.27:0/2704377573 shutdown_connections 2023-12-17T12:08:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.654+0000 7f0c10dff700 1 -- 172.21.15.27:0/2704377573 wait complete. 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.654+0000 7f0c10dff700 1 Processor -- start 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c10dff700 1 -- start start 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c10dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 0x7f0c0c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c10dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c0c102790 0x7f0c0c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c10dff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0c0c103130 0x7f0c0c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c10dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0746a0 con 0x7f0c0c1015a0 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c10dff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c074810 con 0x7f0c0c103130 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c10dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c1a14f0 con 0x7f0c0c102790 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c0a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 0x7f0c0c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c0a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 0x7f0c0c075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38936/0 (socket says 172.21.15.27:38936) 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c0a59c700 1 -- 172.21.15.27:0/3972962988 learned_addr learned my addr 172.21.15.27:0/3972962988 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:55.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c09d9b700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c0c102790 0x7f0c0c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.655+0000 7f0c0ad9d700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0c0c103130 0x7f0c0c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0c0a59c700 1 -- 172.21.15.27:0/3972962988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c0c102790 msgr2=0x7f0c0c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0c0a59c700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c0c102790 0x7f0c0c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0c0a59c700 1 -- 172.21.15.27:0/3972962988 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0c0c103130 msgr2=0x7f0c0c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0c0a59c700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0c0c103130 0x7f0c0c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0c0a59c700 1 -- 172.21.15.27:0/3972962988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc0095f0 con 0x7f0c0c1015a0 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0c0a59c700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 0x7f0c0c075330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009f50 tx=0x7f0bfc003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0bfb7fe700 1 -- 172.21.15.27:0/3972962988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc01d070 con 0x7f0c0c1015a0 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0bfb7fe700 1 -- 172.21.15.27:0/3972962988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bfc00bda0 con 0x7f0c0c1015a0 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0bfb7fe700 1 -- 172.21.15.27:0/3972962988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc022cd0 con 0x7f0c0c1015a0 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c1a1720 con 0x7f0c0c1015a0 2023-12-17T12:08:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.656+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c1a1a90 con 0x7f0c0c1015a0 2023-12-17T12:08:55.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.657+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c108b60 con 0x7f0c0c1015a0 2023-12-17T12:08:55.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.659+0000 7f0bfb7fe700 1 -- 172.21.15.27:0/3972962988 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f0bfc022470 con 0x7f0c0c1015a0 2023-12-17T12:08:55.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1049334969' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-17T12:08:55.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] up:active 2023-12-17T12:08:55.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-17T12:08:55.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-17T12:08:55.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: Cluster is now healthy 2023-12-17T12:08:55.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-17T12:08:55.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-12-17T12:08:55.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-12-17T12:08:55.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3735067277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-17T12:08:55.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.661+0000 7f0bfb7fe700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bf40a1480 0x7f0bf40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:55.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.661+0000 7f0bfb7fe700 1 -- 172.21.15.27:0/3972962988 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0bfc0cdff0 con 0x7f0c0c1015a0 2023-12-17T12:08:55.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.661+0000 7f0c09d9b700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bf40a1480 0x7f0bf40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:55.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.662+0000 7f0c09d9b700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bf40a1480 0x7f0bf40a3940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0c00007c60 tx=0x7f0c000073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:55.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.665+0000 7f0bfb7fe700 1 -- 172.21.15.27:0/3972962988 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0bfc0978f0 con 0x7f0c0c1015a0 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1049334969' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] up:active 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: Cluster is now healthy 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-12-17T12:08:55.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:55 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3735067277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-17T12:08:55.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1049334969' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-17T12:08:55.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] up:active 2023-12-17T12:08:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-17T12:08:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-17T12:08:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: Cluster is now healthy 2023-12-17T12:08:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-17T12:08:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-12-17T12:08:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-12-17T12:08:55.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:55 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3735067277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-17T12:08:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.910+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c0c02d360 con 0x7f0bf40a1480 2023-12-17T12:08:55.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.957+0000 7f0bfb7fe700 1 -- 172.21.15.27:0/3972962988 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0c0c02d360 con 0x7f0bf40a1480 2023-12-17T12:08:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bf40a1480 msgr2=0x7f0bf40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bf40a1480 0x7f0bf40a3940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0c00007c60 tx=0x7f0c000073d0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 msgr2=0x7f0c0c075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 0x7f0c0c075330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009f50 tx=0x7f0bfc003a50 comp rx=0 tx=0).stop 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 shutdown_connections 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bf40a1480 0x7f0bf40a3940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0c0c103130 0x7f0c0c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c0c1015a0 0x7f0c0c075330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 --2- 172.21.15.27:0/3972962988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c0c102790 0x7f0c0c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 >> 172.21.15.27:0/3972962988 conn(0x7f0c0c0fcb10 msgr2=0x7f0c0c105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 shutdown_connections 2023-12-17T12:08:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:55.960+0000 7f0c10dff700 1 -- 172.21.15.27:0/3972962988 wait complete. 2023-12-17T12:08:55.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-12-17T12:08:56.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 -- 172.21.15.27:0/3769545130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 msgr2=0x7f363010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 --2- 172.21.15.27:0/3769545130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 0x7f363010b8f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f362800b0a0 tx=0x7f362800b3b0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 -- 172.21.15.27:0/3769545130 shutdown_connections 2023-12-17T12:08:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 --2- 172.21.15.27:0/3769545130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630075610 0x7f3630073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 --2- 172.21.15.27:0/3769545130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 0x7f363010b8f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 --2- 172.21.15.27:0/3769545130 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3630074030 0x7f36300744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 -- 172.21.15.27:0/3769545130 >> 172.21.15.27:0/3769545130 conn(0x7f36300fcb10 msgr2=0x7f36300fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 -- 172.21.15.27:0/3769545130 shutdown_connections 2023-12-17T12:08:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.190+0000 7f3634fff700 1 -- 172.21.15.27:0/3769545130 wait complete. 2023-12-17T12:08:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.191+0000 7f3634fff700 1 Processor -- start 2023-12-17T12:08:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f3634fff700 1 -- start start 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f3634fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3630074030 0x7f3630199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f3634fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630075610 0x7f3630199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f3634fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 0x7f36301a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f3634fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301a1250 con 0x7f3630109460 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f3634fff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301a1390 con 0x7f3630074030 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f3634fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301a1520 con 0x7f3630075610 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f362ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 0x7f36301a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f362ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 0x7f36301a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59240/0 (socket says 172.21.15.27:59240) 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f362ed9d700 1 -- 172.21.15.27:0/470487320 learned_addr learned my addr 172.21.15.27:0/470487320 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f362dd9b700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630075610 0x7f3630199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.192+0000 7f362e59c700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3630074030 0x7f3630199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f362ed9d700 1 -- 172.21.15.27:0/470487320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630075610 msgr2=0x7f3630199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f362ed9d700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630075610 0x7f3630199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f362ed9d700 1 -- 172.21.15.27:0/470487320 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3630074030 msgr2=0x7f3630199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f362ed9d700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3630074030 0x7f3630199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f362ed9d700 1 -- 172.21.15.27:0/470487320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3628009d00 con 0x7f3630109460 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f362ed9d700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 0x7f36301a0c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f362800b780 tx=0x7f362800bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f361f7fe700 1 -- 172.21.15.27:0/470487320 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362800e040 con 0x7f3630109460 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f361f7fe700 1 -- 172.21.15.27:0/470487320 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f362800be00 con 0x7f3630109460 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f361f7fe700 1 -- 172.21.15.27:0/470487320 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36280052a0 con 0x7f3630109460 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36301a17a0 con 0x7f3630109460 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.193+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36301a1b70 con 0x7f3630109460 2023-12-17T12:08:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.194+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3630108220 con 0x7f3630109460 2023-12-17T12:08:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.196+0000 7f361f7fe700 1 -- 172.21.15.27:0/470487320 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f362801d9a0 con 0x7f3630109460 2023-12-17T12:08:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.198+0000 7f361f7fe700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36180a1500 0x7f36180a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.198+0000 7f361f7fe700 1 -- 172.21.15.27:0/470487320 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36280ce070 con 0x7f3630109460 2023-12-17T12:08:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.198+0000 7f362e59c700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36180a1500 0x7f36180a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.198+0000 7f362e59c700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36180a1500 0x7f36180a39c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3620009bf0 tx=0x7f3620009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.200+0000 7f361f7fe700 1 -- 172.21.15.27:0/470487320 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3628097810 con 0x7f3630109460 2023-12-17T12:08:56.411 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:08:55.917+0000 7fce80f2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-17T12:08:56.412 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:08:55.917+0000 7fce80f2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-17T12:08:56.412 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:08:55.917+0000 7fce80f2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-17T12:08:56.412 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:08:55.917+0000 7fce80f2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-17T12:08:56.412 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:08:55 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:08:55.917+0000 7fce80f2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-17T12:08:56.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.452+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f363002d380 con 0x7f36180a1500 2023-12-17T12:08:56.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.457+0000 7f361f7fe700 1 -- 172.21.15.27:0/470487320 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f363002d380 con 0x7f36180a1500 2023-12-17T12:08:56.458 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/sv_0/aa3dffb6-cf4f-452f-b2d3-a89bb7f89aea 2023-12-17T12:08:56.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.459+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36180a1500 msgr2=0x7f36180a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.459+0000 7f3634fff700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36180a1500 0x7f36180a39c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3620009bf0 tx=0x7f3620009260 comp rx=0 tx=0).stop 2023-12-17T12:08:56.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.459+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 msgr2=0x7f36301a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.459+0000 7f3634fff700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 0x7f36301a0c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f362800b780 tx=0x7f362800bb60 comp rx=0 tx=0).stop 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.460+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 shutdown_connections 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.460+0000 7f3634fff700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f36180a1500 0x7f36180a39c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.460+0000 7f3634fff700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3630109460 0x7f36301a0c80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.460+0000 7f3634fff700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3630074030 0x7f3630199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.460+0000 7f3634fff700 1 --2- 172.21.15.27:0/470487320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630075610 0x7f3630199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.460+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 >> 172.21.15.27:0/470487320 conn(0x7f36300fcb10 msgr2=0x7f3630105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.460+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 shutdown_connections 2023-12-17T12:08:56.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.460+0000 7f3634fff700 1 -- 172.21.15.27:0/470487320 wait complete. 2023-12-17T12:08:56.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-12-17T12:08:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.691+0000 7f2b733ff700 1 -- 172.21.15.27:0/1463972735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 msgr2=0x7f2b6c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.691+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1463972735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 0x7f2b6c102bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0099c0 tx=0x7f2b5c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.691+0000 7f2b733ff700 1 -- 172.21.15.27:0/1463972735 shutdown_connections 2023-12-17T12:08:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.691+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1463972735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b6c103130 0x7f2b6c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.691+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1463972735 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2b6c1015a0 0x7f2b6c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.691+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1463972735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 0x7f2b6c102bf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.691+0000 7f2b733ff700 1 -- 172.21.15.27:0/1463972735 >> 172.21.15.27:0/1463972735 conn(0x7f2b6c0fcb10 msgr2=0x7f2b6c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.691+0000 7f2b733ff700 1 -- 172.21.15.27:0/1463972735 shutdown_connections 2023-12-17T12:08:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.692+0000 7f2b733ff700 1 -- 172.21.15.27:0/1463972735 wait complete. 2023-12-17T12:08:56.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.692+0000 7f2b733ff700 1 Processor -- start 2023-12-17T12:08:56.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b733ff700 1 -- start start 2023-12-17T12:08:56.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b733ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b6c1015a0 0x7f2b6c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:56.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b733ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 0x7f2b6c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b733ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2b6c103130 0x7f2b6c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b733ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1a1140 con 0x7f2b6c102790 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b733ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1a1280 con 0x7f2b6c103130 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b733ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1a1410 con 0x7f2b6c1015a0 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b7099a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 0x7f2b6c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b7099a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 0x7f2b6c199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59264/0 (socket says 172.21.15.27:59264) 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b7099a700 1 -- 172.21.15.27:0/1295380088 learned_addr learned my addr 172.21.15.27:0/1295380088 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.693+0000 7f2b7119b700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b6c1015a0 0x7f2b6c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b7199c700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2b6c103130 0x7f2b6c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:56.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b7099a700 1 -- 172.21.15.27:0/1295380088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b6c1015a0 msgr2=0x7f2b6c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b7099a700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b6c1015a0 0x7f2b6c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b7099a700 1 -- 172.21.15.27:0/1295380088 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2b6c103130 msgr2=0x7f2b6c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b7099a700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2b6c103130 0x7f2b6c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b7099a700 1 -- 172.21.15.27:0/1295380088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c0095f0 con 0x7f2b6c102790 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b7199c700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2b6c103130 0x7f2b6c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b7099a700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 0x7f2b6c199ba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009990 tx=0x7f2b5c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b627fc700 1 -- 172.21.15.27:0/1295380088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01d070 con 0x7f2b6c102790 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b627fc700 1 -- 172.21.15.27:0/1295380088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b5c00bd10 con 0x7f2b6c102790 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6c1a1690 con 0x7f2b6c102790 2023-12-17T12:08:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b627fc700 1 -- 172.21.15.27:0/1295380088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c017650 con 0x7f2b6c102790 2023-12-17T12:08:56.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.694+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c1a1ab0 con 0x7f2b6c102790 2023-12-17T12:08:56.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.696+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6c068310 con 0x7f2b6c102790 2023-12-17T12:08:56.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.698+0000 7f2b627fc700 1 -- 172.21.15.27:0/1295380088 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2b5c022470 con 0x7f2b6c102790 2023-12-17T12:08:56.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.699+0000 7f2b627fc700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2b580a1480 0x7f2b580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:56.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.699+0000 7f2b627fc700 1 -- 172.21.15.27:0/1295380088 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2b5c0cdf40 con 0x7f2b6c102790 2023-12-17T12:08:56.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.699+0000 7f2b7119b700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2b580a1480 0x7f2b580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:56.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.700+0000 7f2b7119b700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2b580a1480 0x7f2b580a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c1025f0 tx=0x7f2b68009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.703+0000 7f2b627fc700 1 -- 172.21.15.27:0/1295380088 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2b5c0978a0 con 0x7f2b6c102790 2023-12-17T12:08:56.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:56 smithi141 ceph-mon[41225]: pgmap v268: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s wr, 12 op/s 2023-12-17T12:08:56.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:56 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3735067277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-17T12:08:56.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:56 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] up:active 2023-12-17T12:08:56.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:56 smithi141 ceph-mon[41225]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-12-17T12:08:56.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:56 smithi141 ceph-mon[41225]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-17T12:08:56.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:56 smithi192 ceph-mon[41899]: pgmap v268: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s wr, 12 op/s 2023-12-17T12:08:56.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:56 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3735067277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-17T12:08:56.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:56 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] up:active 2023-12-17T12:08:56.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:56 smithi192 ceph-mon[41899]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-12-17T12:08:56.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:56 smithi192 ceph-mon[41899]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-17T12:08:56.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:56 smithi027 ceph-mon[36871]: pgmap v268: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s wr, 12 op/s 2023-12-17T12:08:56.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:56 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3735067277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-17T12:08:56.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:56 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] up:active 2023-12-17T12:08:56.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:56 smithi027 ceph-mon[36871]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-12-17T12:08:56.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:56 smithi027 ceph-mon[36871]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-17T12:08:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.951+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b6c074f20 con 0x7f2b580a1480 2023-12-17T12:08:56.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.979+0000 7f2b627fc700 1 -- 172.21.15.27:0/1295380088 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2b6c074f20 con 0x7f2b580a1480 2023-12-17T12:08:56.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2b580a1480 msgr2=0x7f2b580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2b580a1480 0x7f2b580a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c1025f0 tx=0x7f2b68009260 comp rx=0 tx=0).stop 2023-12-17T12:08:56.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 msgr2=0x7f2b6c199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:56.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 0x7f2b6c199ba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009990 tx=0x7f2b5c00fa00 comp rx=0 tx=0).stop 2023-12-17T12:08:56.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 shutdown_connections 2023-12-17T12:08:56.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2b580a1480 0x7f2b580a3940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b6c1015a0 0x7f2b6c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2b6c103130 0x7f2b6c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.981+0000 7f2b733ff700 1 --2- 172.21.15.27:0/1295380088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c102790 0x7f2b6c199ba0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:56.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.982+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 >> 172.21.15.27:0/1295380088 conn(0x7f2b6c0fcb10 msgr2=0x7f2b6c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:56.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.982+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 shutdown_connections 2023-12-17T12:08:56.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:56.982+0000 7f2b733ff700 1 -- 172.21.15.27:0/1295380088 wait complete. 2023-12-17T12:08:57.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-12-17T12:08:57.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.216+0000 7fde8b39d700 1 -- 172.21.15.27:0/4255915682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841025d0 msgr2=0x7fde84102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.216+0000 7fde8b39d700 1 --2- 172.21.15.27:0/4255915682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841025d0 0x7fde84102a30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fde740099c0 tx=0x7fde74009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.216+0000 7fde8b39d700 1 -- 172.21.15.27:0/4255915682 shutdown_connections 2023-12-17T12:08:57.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.216+0000 7fde8b39d700 1 --2- 172.21.15.27:0/4255915682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde84102f70 0x7fde84109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.216+0000 7fde8b39d700 1 --2- 172.21.15.27:0/4255915682 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fde841013e0 0x7fde84101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.216+0000 7fde8b39d700 1 --2- 172.21.15.27:0/4255915682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841025d0 0x7fde84102a30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.217+0000 7fde8b39d700 1 -- 172.21.15.27:0/4255915682 >> 172.21.15.27:0/4255915682 conn(0x7fde840fc930 msgr2=0x7fde840fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:57.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.217+0000 7fde8b39d700 1 -- 172.21.15.27:0/4255915682 shutdown_connections 2023-12-17T12:08:57.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.217+0000 7fde8b39d700 1 -- 172.21.15.27:0/4255915682 wait complete. 2023-12-17T12:08:57.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8b39d700 1 Processor -- start 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8b39d700 1 -- start start 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8b39d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841013e0 0x7fde841994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8b39d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fde841025d0 0x7fde84199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8b39d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde84102f70 0x7fde841a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8b39d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde841a0fa0 con 0x7fde841013e0 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8b39d700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde841a10e0 con 0x7fde841025d0 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8b39d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde841a1270 con 0x7fde84102f70 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8a39b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841013e0 0x7fde841994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8a39b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841013e0 0x7fde841994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59294/0 (socket says 172.21.15.27:59294) 2023-12-17T12:08:57.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.218+0000 7fde8a39b700 1 -- 172.21.15.27:0/3646058887 learned_addr learned my addr 172.21.15.27:0/3646058887 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde89b9a700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fde841025d0 0x7fde84199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde8a39b700 1 -- 172.21.15.27:0/3646058887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde84102f70 msgr2=0x7fde841a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde8a39b700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde84102f70 0x7fde841a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde8a39b700 1 -- 172.21.15.27:0/3646058887 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fde841025d0 msgr2=0x7fde84199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde8a39b700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fde841025d0 0x7fde84199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde8a39b700 1 -- 172.21.15.27:0/3646058887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde740095f0 con 0x7fde841013e0 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde89b9a700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fde841025d0 0x7fde84199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde8a39b700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841013e0 0x7fde841994c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fde8000e8f0 tx=0x7fde8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde7b7fe700 1 -- 172.21.15.27:0/3646058887 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80009790 con 0x7fde841013e0 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde7b7fe700 1 -- 172.21.15.27:0/3646058887 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde800044b0 con 0x7fde841013e0 2023-12-17T12:08:57.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde841a1550 con 0x7fde841013e0 2023-12-17T12:08:57.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde7b7fe700 1 -- 172.21.15.27:0/3646058887 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80003ce0 con 0x7fde841013e0 2023-12-17T12:08:57.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.219+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde841a1ab0 con 0x7fde841013e0 2023-12-17T12:08:57.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.221+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde840681c0 con 0x7fde841013e0 2023-12-17T12:08:57.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.223+0000 7fde7b7fe700 1 -- 172.21.15.27:0/3646058887 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fde80003e40 con 0x7fde841013e0 2023-12-17T12:08:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.224+0000 7fde7b7fe700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fde700a1500 0x7fde700a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.224+0000 7fde7b7fe700 1 -- 172.21.15.27:0/3646058887 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fde80014070 con 0x7fde841013e0 2023-12-17T12:08:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.224+0000 7fde89b9a700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fde700a1500 0x7fde700a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.225+0000 7fde89b9a700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fde700a1500 0x7fde700a39c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde740098e0 tx=0x7fde7400b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:57.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.228+0000 7fde7b7fe700 1 -- 172.21.15.27:0/3646058887 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fde80096560 con 0x7fde841013e0 2023-12-17T12:08:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.437+0000 7fde7b7fe700 1 -- 172.21.15.27:0/3646058887 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fde80096930 con 0x7fde841013e0 2023-12-17T12:08:57.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.484+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fde84074f20 con 0x7fde700a1500 2023-12-17T12:08:57.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.489+0000 7fde7b7fe700 1 -- 172.21.15.27:0/3646058887 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fde84074f20 con 0x7fde700a1500 2023-12-17T12:08:57.490 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/sv_1/f391da7c-ecc1-4b26-93d1-9dbe0616870e 2023-12-17T12:08:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fde700a1500 msgr2=0x7fde700a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fde700a1500 0x7fde700a39c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde740098e0 tx=0x7fde7400b540 comp rx=0 tx=0).stop 2023-12-17T12:08:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841013e0 msgr2=0x7fde841994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841013e0 0x7fde841994c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fde8000e8f0 tx=0x7fde8000ecb0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 shutdown_connections 2023-12-17T12:08:57.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fde700a1500 0x7fde700a39c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde84102f70 0x7fde841a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde841013e0 0x7fde841994c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.491+0000 7fde8b39d700 1 --2- 172.21.15.27:0/3646058887 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fde841025d0 0x7fde84199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.492+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 >> 172.21.15.27:0/3646058887 conn(0x7fde840fc930 msgr2=0x7fde84073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:57.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.492+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 shutdown_connections 2023-12-17T12:08:57.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.492+0000 7fde8b39d700 1 -- 172.21.15.27:0/3646058887 wait complete. 2023-12-17T12:08:57.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:08:57.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 -- 172.21.15.27:0/2846063796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4104830 msgr2=0x7f26a4106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 --2- 172.21.15.27:0/2846063796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4104830 0x7f26a4106c20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2698009990 tx=0x7f2698009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 -- 172.21.15.27:0/2846063796 shutdown_connections 2023-12-17T12:08:57.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 --2- 172.21.15.27:0/2846063796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a4101f00 0x7f26a41042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 --2- 172.21.15.27:0/2846063796 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f26a4107160 0x7f26a41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 --2- 172.21.15.27:0/2846063796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4104830 0x7f26a4106c20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 -- 172.21.15.27:0/2846063796 >> 172.21.15.27:0/2846063796 conn(0x7f26a40fb8f0 msgr2=0x7f26a40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:57.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 -- 172.21.15.27:0/2846063796 shutdown_connections 2023-12-17T12:08:57.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.719+0000 7f26a91ff700 1 -- 172.21.15.27:0/2846063796 wait complete. 2023-12-17T12:08:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.720+0000 7f26a91ff700 1 Processor -- start 2023-12-17T12:08:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.720+0000 7f26a91ff700 1 -- start start 2023-12-17T12:08:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a91ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4101f00 0x7f26a4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a91ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f26a4104830 0x7f26a4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:57.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a91ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a4107160 0x7f26a419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a91ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a419ee80 con 0x7f26a4101f00 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a91ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a419efc0 con 0x7f26a4104830 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a91ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a419f150 con 0x7f26a4107160 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4101f00 0x7f26a4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4101f00 0x7f26a4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59318/0 (socket says 172.21.15.27:59318) 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a2d9d700 1 -- 172.21.15.27:0/4264311557 learned_addr learned my addr 172.21.15.27:0/4264311557 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a259c700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f26a4104830 0x7f26a4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a359e700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a4107160 0x7f26a419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a2d9d700 1 -- 172.21.15.27:0/4264311557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a4107160 msgr2=0x7f26a419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a2d9d700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a4107160 0x7f26a419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a2d9d700 1 -- 172.21.15.27:0/4264311557 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f26a4104830 msgr2=0x7f26a4197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a2d9d700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f26a4104830 0x7f26a4197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a2d9d700 1 -- 172.21.15.27:0/4264311557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26980095f0 con 0x7f26a4101f00 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a259c700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f26a4104830 0x7f26a4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.721+0000 7f26a359e700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a4107160 0x7f26a419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.722+0000 7f26a2d9d700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4101f00 0x7f26a4197350 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f269400e9a0 tx=0x7f269400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.722+0000 7f268bfff700 1 -- 172.21.15.27:0/4264311557 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694009790 con 0x7f26a4101f00 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.722+0000 7f268bfff700 1 -- 172.21.15.27:0/4264311557 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f269400cbd0 con 0x7f26a4101f00 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.722+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a419f430 con 0x7f26a4101f00 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.722+0000 7f268bfff700 1 -- 172.21.15.27:0/4264311557 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694018680 con 0x7f26a4101f00 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.722+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a419f940 con 0x7f26a4101f00 2023-12-17T12:08:57.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.723+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a40fd470 con 0x7f26a4101f00 2023-12-17T12:08:57.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.725+0000 7f268bfff700 1 -- 172.21.15.27:0/4264311557 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f269400cd40 con 0x7f26a4101f00 2023-12-17T12:08:57.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.727+0000 7f268bfff700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f268c0a1510 0x7f268c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:57.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.728+0000 7f268bfff700 1 -- 172.21.15.27:0/4264311557 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2694014070 con 0x7f26a4101f00 2023-12-17T12:08:57.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.728+0000 7f26a259c700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f268c0a1510 0x7f268c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:57.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.728+0000 7f26a259c700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f268c0a1510 0x7f268c0a39d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2698006010 tx=0x7f269800b630 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.732+0000 7f268bfff700 1 -- 172.21.15.27:0/4264311557 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f26940522f0 con 0x7f26a4101f00 2023-12-17T12:08:57.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:57 smithi141 ceph-mon[41225]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:57.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:57 smithi141 ceph-mon[41225]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:57.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:57 smithi192 ceph-mon[41899]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:57.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:57 smithi192 ceph-mon[41899]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:57.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:57 smithi027 ceph-mon[36871]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:57.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:57 smithi027 ceph-mon[36871]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:57.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.990+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26a4068230 con 0x7f26a4101f00 2023-12-17T12:08:57.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.991+0000 7f268bfff700 1 -- 172.21.15.27:0/4264311557 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11327 (secure 0 0 0) 0x7f269409a120 con 0x7f26a4101f00 2023-12-17T12:08:57.993 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:57.993 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3551652066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3551652066},{"type":"v1","addr":"172.21.15.27:6841","nonce":3551652066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-17T12:08:48.917098+0000","modified":"2023-12-17T12:08:55.420903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24538,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.141:6839/3282544216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6838","nonce":3282544216},{"type":"v1","addr":"172.21.15.141:6839","nonce":3282544216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-17T12:08:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f268c0a1510 msgr2=0x7f268c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f268c0a1510 0x7f268c0a39d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2698006010 tx=0x7f269800b630 comp rx=0 tx=0).stop 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4101f00 msgr2=0x7f26a4197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4101f00 0x7f26a4197350 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f269400e9a0 tx=0x7f269400ed60 comp rx=0 tx=0).stop 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 shutdown_connections 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f268c0a1510 0x7f268c0a39d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a4107160 0x7f26a419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a4101f00 0x7f26a4197350 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 --2- 172.21.15.27:0/4264311557 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f26a4104830 0x7f26a4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 >> 172.21.15.27:0/4264311557 conn(0x7f26a40fb8f0 msgr2=0x7f26a4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 shutdown_connections 2023-12-17T12:08:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:57.994+0000 7f26a91ff700 1 -- 172.21.15.27:0/4264311557 wait complete. 2023-12-17T12:08:57.997 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-17T12:08:58.015 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:08:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 -- 172.21.15.27:0/2813640300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa940ffcf0 msgr2=0x7ffa94102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2813640300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa940ffcf0 0x7ffa94102120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009990 tx=0x7ffa84009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 -- 172.21.15.27:0/2813640300 shutdown_connections 2023-12-17T12:08:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2813640300 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ffa94105030 0x7ffa941054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2813640300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa940ffcf0 0x7ffa94102120 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2813640300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa94102660 0x7ffa94104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 -- 172.21.15.27:0/2813640300 >> 172.21.15.27:0/2813640300 conn(0x7ffa940fb8f0 msgr2=0x7ffa940fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 -- 172.21.15.27:0/2813640300 shutdown_connections 2023-12-17T12:08:58.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.221+0000 7ffa999ff700 1 -- 172.21.15.27:0/2813640300 wait complete. 2023-12-17T12:08:58.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.222+0000 7ffa999ff700 1 Processor -- start 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.222+0000 7ffa999ff700 1 -- start start 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa999ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa940ffcf0 0x7ffa9419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa999ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa94102660 0x7ffa9419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa999ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ffa94105030 0x7ffa941a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa999ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941a5470 con 0x7ffa94102660 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa999ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941a55b0 con 0x7ffa94105030 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa999ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941a5740 con 0x7ffa940ffcf0 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa927fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa94102660 0x7ffa9419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa927fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa94102660 0x7ffa9419de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59338/0 (socket says 172.21.15.27:59338) 2023-12-17T12:08:58.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa927fc700 1 -- 172.21.15.27:0/2385363428 learned_addr learned my addr 172.21.15.27:0/2385363428 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa92ffd700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa940ffcf0 0x7ffa9419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa937fe700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ffa94105030 0x7ffa941a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa927fc700 1 -- 172.21.15.27:0/2385363428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa940ffcf0 msgr2=0x7ffa9419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa927fc700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa940ffcf0 0x7ffa9419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa927fc700 1 -- 172.21.15.27:0/2385363428 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ffa94105030 msgr2=0x7ffa941a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa927fc700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ffa94105030 0x7ffa941a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa927fc700 1 -- 172.21.15.27:0/2385363428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840095f0 con 0x7ffa94102660 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.223+0000 7ffa92ffd700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa940ffcf0 0x7ffa9419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.224+0000 7ffa927fc700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa94102660 0x7ffa9419de80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800e9a0 tx=0x7ffa8800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.224+0000 7ffa989fd700 1 -- 172.21.15.27:0/2385363428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88009790 con 0x7ffa94102660 2023-12-17T12:08:58.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.224+0000 7ffa989fd700 1 -- 172.21.15.27:0/2385363428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa8800cbd0 con 0x7ffa94102660 2023-12-17T12:08:58.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.224+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa941a5a20 con 0x7ffa94102660 2023-12-17T12:08:58.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.224+0000 7ffa989fd700 1 -- 172.21.15.27:0/2385363428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88018680 con 0x7ffa94102660 2023-12-17T12:08:58.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.224+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa941a5f30 con 0x7ffa94102660 2023-12-17T12:08:58.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.225+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa940fd470 con 0x7ffa94102660 2023-12-17T12:08:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.227+0000 7ffa989fd700 1 -- 172.21.15.27:0/2385363428 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ffa8800cd40 con 0x7ffa94102660 2023-12-17T12:08:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.228+0000 7ffa989fd700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ffa7c0a1510 0x7ffa7c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.229+0000 7ffa989fd700 1 -- 172.21.15.27:0/2385363428 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffa88014070 con 0x7ffa94102660 2023-12-17T12:08:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.229+0000 7ffa92ffd700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ffa7c0a1510 0x7ffa7c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.229+0000 7ffa92ffd700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ffa7c0a1510 0x7ffa7c0a39d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400b670 tx=0x7ffa84005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.233+0000 7ffa989fd700 1 -- 172.21.15.27:0/2385363428 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ffa880522f0 con 0x7ffa94102660 2023-12-17T12:08:58.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.448+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa94113160 con 0x7ffa94102660 2023-12-17T12:08:58.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.449+0000 7ffa989fd700 1 -- 172.21.15.27:0/2385363428 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22130 (secure 0 0 0) 0x7ffa8809a120 con 0x7ffa94102660 2023-12-17T12:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:08:50.161605+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:08:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.451+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ffa7c0a1510 msgr2=0x7ffa7c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.451+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ffa7c0a1510 0x7ffa7c0a39d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400b670 tx=0x7ffa84005fb0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.451+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa94102660 msgr2=0x7ffa9419de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa94102660 0x7ffa9419de80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800e9a0 tx=0x7ffa8800ed60 comp rx=0 tx=0).stop 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 shutdown_connections 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ffa7c0a1510 0x7ffa7c0a39d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa940ffcf0 0x7ffa9419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ffa94105030 0x7ffa941a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 --2- 172.21.15.27:0/2385363428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa94102660 0x7ffa9419de80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 >> 172.21.15.27:0/2385363428 conn(0x7ffa940fb8f0 msgr2=0x7ffa94103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 shutdown_connections 2023-12-17T12:08:58.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.452+0000 7ffa999ff700 1 -- 172.21.15.27:0/2385363428 wait complete. 2023-12-17T12:08:58.472 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-17T12:08:58.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-17T12:08:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 -- 172.21.15.27:0/1375050367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541048a0 msgr2=0x7f7654104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 --2- 172.21.15.27:0/1375050367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541048a0 0x7f7654104d00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f76480099c0 tx=0x7f7648009cd0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 -- 172.21.15.27:0/1375050367 shutdown_connections 2023-12-17T12:08:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 --2- 172.21.15.27:0/1375050367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7654105240 0x7f765410baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 --2- 172.21.15.27:0/1375050367 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f76541036b0 0x7f7654103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 --2- 172.21.15.27:0/1375050367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541048a0 0x7f7654104d00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 -- 172.21.15.27:0/1375050367 >> 172.21.15.27:0/1375050367 conn(0x7f76540fec40 msgr2=0x7f7654101060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 -- 172.21.15.27:0/1375050367 shutdown_connections 2023-12-17T12:08:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.680+0000 7f7659dff700 1 -- 172.21.15.27:0/1375050367 wait complete. 2023-12-17T12:08:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.682+0000 7f7659dff700 1 Processor -- start 2023-12-17T12:08:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.682+0000 7f7659dff700 1 -- start start 2023-12-17T12:08:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.682+0000 7f7659dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541036b0 0x7f7654199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.682+0000 7f7659dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76541048a0 0x7f7654199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.682+0000 7f7659dff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7654105240 0x7f76541a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.682+0000 7f7659dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76541a1090 con 0x7f76541036b0 2023-12-17T12:08:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.682+0000 7f7659dff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76541a11d0 con 0x7f7654105240 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.682+0000 7f7659dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76541a1360 con 0x7f76541048a0 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541036b0 0x7f7654199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541036b0 0x7f7654199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59350/0 (socket says 172.21.15.27:59350) 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 -- 172.21.15.27:0/2175313268 learned_addr learned my addr 172.21.15.27:0/2175313268 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f7652ffd700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76541048a0 0x7f7654199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f7653fff700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7654105240 0x7f76541a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 -- 172.21.15.27:0/2175313268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76541048a0 msgr2=0x7f7654199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76541048a0 0x7f7654199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 -- 172.21.15.27:0/2175313268 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7654105240 msgr2=0x7f76541a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7654105240 0x7f76541a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 -- 172.21.15.27:0/2175313268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480095f0 con 0x7f76541036b0 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f7653fff700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7654105240 0x7f76541a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.683+0000 7f76537fe700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541036b0 0x7f7654199560 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f764400d980 tx=0x7f764400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.684+0000 7f7650ff9700 1 -- 172.21.15.27:0/2175313268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400b950 con 0x7f76541036b0 2023-12-17T12:08:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.684+0000 7f7650ff9700 1 -- 172.21.15.27:0/2175313268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7644007620 con 0x7f76541036b0 2023-12-17T12:08:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.684+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76541a1640 con 0x7f76541036b0 2023-12-17T12:08:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.684+0000 7f7650ff9700 1 -- 172.21.15.27:0/2175313268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400f3c0 con 0x7f76541036b0 2023-12-17T12:08:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.684+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76541a1ba0 con 0x7f76541036b0 2023-12-17T12:08:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.685+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765410ab30 con 0x7f76541036b0 2023-12-17T12:08:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.687+0000 7f7650ff9700 1 -- 172.21.15.27:0/2175313268 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f764400bab0 con 0x7f76541036b0 2023-12-17T12:08:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.688+0000 7f7650ff9700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f763c0a1590 0x7f763c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.688+0000 7f7650ff9700 1 -- 172.21.15.27:0/2175313268 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f76440ccfa0 con 0x7f76541036b0 2023-12-17T12:08:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.688+0000 7f7652ffd700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f763c0a1590 0x7f763c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:58.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.689+0000 7f7652ffd700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f763c0a1590 0x7f763c0a3a50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7648000c00 tx=0x7f7648003680 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:58.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.692+0000 7f7650ff9700 1 -- 172.21.15.27:0/2175313268 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7644096550 con 0x7f76541036b0 2023-12-17T12:08:58.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:58 smithi141 ceph-mon[41225]: pgmap v269: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2023-12-17T12:08:58.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:58 smithi141 ceph-mon[41225]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:58.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:58 smithi141 ceph-mon[41225]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-12-17T12:08:58.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:58 smithi141 ceph-mon[41225]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:58.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:58 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4264311557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:58.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:58 smithi192 ceph-mon[41899]: pgmap v269: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2023-12-17T12:08:58.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:58 smithi192 ceph-mon[41899]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:58.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:58 smithi192 ceph-mon[41899]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-12-17T12:08:58.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:58 smithi192 ceph-mon[41899]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:58.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:58 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4264311557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.907+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f765402d380 con 0x7f763c0a1590 2023-12-17T12:08:58.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:58 smithi027 ceph-mon[36871]: pgmap v269: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2023-12-17T12:08:58.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:58 smithi027 ceph-mon[36871]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:58.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:58 smithi027 ceph-mon[36871]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-12-17T12:08:58.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:58 smithi027 ceph-mon[36871]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:08:58.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:58 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4264311557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:08:58.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.920+0000 7f7650ff9700 1 -- 172.21.15.27:0/2175313268 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334461 (secure 0 0 0) 0x7f765402d380 con 0x7f763c0a1590 2023-12-17T12:08:58.927 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f763c0a1590 msgr2=0x7f763c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f763c0a1590 0x7f763c0a3a50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7648000c00 tx=0x7f7648003680 comp rx=0 tx=0).stop 2023-12-17T12:08:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541036b0 msgr2=0x7f7654199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541036b0 0x7f7654199560 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f764400d980 tx=0x7f764400dc90 comp rx=0 tx=0).stop 2023-12-17T12:08:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 shutdown_connections 2023-12-17T12:08:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f763c0a1590 0x7f763c0a3a50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7654105240 0x7f76541a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76541036b0 0x7f7654199560 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.925+0000 7f7659dff700 1 --2- 172.21.15.27:0/2175313268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76541048a0 0x7f7654199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.926+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 >> 172.21.15.27:0/2175313268 conn(0x7f76540fec40 msgr2=0x7f7654107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.926+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 shutdown_connections 2023-12-17T12:08:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:58.926+0000 7f7659dff700 1 -- 172.21.15.27:0/2175313268 wait complete. 2023-12-17T12:08:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-17T12:08:58.946 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2023-12-17T12:08:58.796718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596102,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":155,"num_read_kb":133,"num_write":304,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334896,"kb_used_data":10704,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606928,"statfs":{"total":1151940427776,"available":1151597494272,"internally_reserved":0,"allocated":10960896,"data_stored":6893078,"data_compressed":107256,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19139,"internal_metadata":331855165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5734,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":123,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.634281"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801249+0000","last_change":"2023-12-17T12:08:49.184999+0000","last_active":"2023-12-17T12:08:50.801249+0000","last_peered":"2023-12-17T12:08:50.801249+0000","last_clean":"2023-12-17T12:08:50.801249+0000","last_became_active":"2023-12-17T12:08:49.184214+0000","last_became_peered":"2023-12-17T12:08:49.184214+0000","last_unstale":"2023-12-17T12:08:50.801249+0000","last_undegraded":"2023-12-17T12:08:50.801249+0000","last_fullsized":"2023-12-17T12:08:50.801249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:48:56.174610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799801+0000","last_change":"2023-12-17T12:08:49.188212+0000","last_active":"2023-12-17T12:08:50.799801+0000","last_peered":"2023-12-17T12:08:50.799801+0000","last_clean":"2023-12-17T12:08:50.799801+0000","last_became_active":"2023-12-17T12:08:49.187869+0000","last_became_peered":"2023-12-17T12:08:49.187869+0000","last_unstale":"2023-12-17T12:08:50.799801+0000","last_undegraded":"2023-12-17T12:08:50.799801+0000","last_fullsized":"2023-12-17T12:08:50.799801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:53:52.485169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799658+0000","last_change":"2023-12-17T12:08:49.182801+0000","last_active":"2023-12-17T12:08:50.799658+0000","last_peered":"2023-12-17T12:08:50.799658+0000","last_clean":"2023-12-17T12:08:50.799658+0000","last_became_active":"2023-12-17T12:08:49.182488+0000","last_became_peered":"2023-12-17T12:08:49.182488+0000","last_unstale":"2023-12-17T12:08:50.799658+0000","last_undegraded":"2023-12-17T12:08:50.799658+0000","last_fullsized":"2023-12-17T12:08:50.799658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:16:43.566881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166805+0000","last_change":"2023-12-17T12:08:49.189279+0000","last_active":"2023-12-17T12:08:50.166805+0000","last_peered":"2023-12-17T12:08:50.166805+0000","last_clean":"2023-12-17T12:08:50.166805+0000","last_became_active":"2023-12-17T12:08:49.186419+0000","last_became_peered":"2023-12-17T12:08:49.186419+0000","last_unstale":"2023-12-17T12:08:50.166805+0000","last_undegraded":"2023-12-17T12:08:50.166805+0000","last_fullsized":"2023-12-17T12:08:50.166805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:47:03.035599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166781+0000","last_change":"2023-12-17T12:08:49.192587+0000","last_active":"2023-12-17T12:08:50.166781+0000","last_peered":"2023-12-17T12:08:50.166781+0000","last_clean":"2023-12-17T12:08:50.166781+0000","last_became_active":"2023-12-17T12:08:49.192272+0000","last_became_peered":"2023-12-17T12:08:49.192272+0000","last_unstale":"2023-12-17T12:08:50.166781+0000","last_undegraded":"2023-12-17T12:08:50.166781+0000","last_fullsized":"2023-12-17T12:08:50.166781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:45:30.131065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801258+0000","last_change":"2023-12-17T12:08:49.176906+0000","last_active":"2023-12-17T12:08:50.801258+0000","last_peered":"2023-12-17T12:08:50.801258+0000","last_clean":"2023-12-17T12:08:50.801258+0000","last_became_active":"2023-12-17T12:08:49.176591+0000","last_became_peered":"2023-12-17T12:08:49.176591+0000","last_unstale":"2023-12-17T12:08:50.801258+0000","last_undegraded":"2023-12-17T12:08:50.801258+0000","last_fullsized":"2023-12-17T12:08:50.801258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:27:17.508628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800023+0000","last_change":"2023-12-17T12:08:49.177409+0000","last_active":"2023-12-17T12:08:50.800023+0000","last_peered":"2023-12-17T12:08:50.800023+0000","last_clean":"2023-12-17T12:08:50.800023+0000","last_became_active":"2023-12-17T12:08:49.177093+0000","last_became_peered":"2023-12-17T12:08:49.177093+0000","last_unstale":"2023-12-17T12:08:50.800023+0000","last_undegraded":"2023-12-17T12:08:50.800023+0000","last_fullsized":"2023-12-17T12:08:50.800023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:56:30.669406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799992+0000","last_change":"2023-12-17T12:08:49.194035+0000","last_active":"2023-12-17T12:08:50.799992+0000","last_peered":"2023-12-17T12:08:50.799992+0000","last_clean":"2023-12-17T12:08:50.799992+0000","last_became_active":"2023-12-17T12:08:49.191677+0000","last_became_peered":"2023-12-17T12:08:49.191677+0000","last_unstale":"2023-12-17T12:08:50.799992+0000","last_undegraded":"2023-12-17T12:08:50.799992+0000","last_fullsized":"2023-12-17T12:08:50.799992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:21:00.984977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800536+0000","last_change":"2023-12-17T12:08:49.187140+0000","last_active":"2023-12-17T12:08:50.800536+0000","last_peered":"2023-12-17T12:08:50.800536+0000","last_clean":"2023-12-17T12:08:50.800536+0000","last_became_active":"2023-12-17T12:08:49.186807+0000","last_became_peered":"2023-12-17T12:08:49.186807+0000","last_unstale":"2023-12-17T12:08:50.800536+0000","last_undegraded":"2023-12-17T12:08:50.800536+0000","last_fullsized":"2023-12-17T12:08:50.800536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:28:59.060986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801544+0000","last_change":"2023-12-17T12:08:49.186371+0000","last_active":"2023-12-17T12:08:50.801544+0000","last_peered":"2023-12-17T12:08:50.801544+0000","last_clean":"2023-12-17T12:08:50.801544+0000","last_became_active":"2023-12-17T12:08:49.186045+0000","last_became_peered":"2023-12-17T12:08:49.186045+0000","last_unstale":"2023-12-17T12:08:50.801544+0000","last_undegraded":"2023-12-17T12:08:50.801544+0000","last_fullsized":"2023-12-17T12:08:50.801544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:56:47.851617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800846+0000","last_change":"2023-12-17T12:08:49.179088+0000","last_active":"2023-12-17T12:08:50.800846+0000","last_peered":"2023-12-17T12:08:50.800846+0000","last_clean":"2023-12-17T12:08:50.800846+0000","last_became_active":"2023-12-17T12:08:49.178545+0000","last_became_peered":"2023-12-17T12:08:49.178545+0000","last_unstale":"2023-12-17T12:08:50.800846+0000","last_undegraded":"2023-12-17T12:08:50.800846+0000","last_fullsized":"2023-12-17T12:08:50.800846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:52:58.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166723+0000","last_change":"2023-12-17T12:08:49.187681+0000","last_active":"2023-12-17T12:08:50.166723+0000","last_peered":"2023-12-17T12:08:50.166723+0000","last_clean":"2023-12-17T12:08:50.166723+0000","last_became_active":"2023-12-17T12:08:49.187086+0000","last_became_peered":"2023-12-17T12:08:49.187086+0000","last_unstale":"2023-12-17T12:08:50.166723+0000","last_undegraded":"2023-12-17T12:08:50.166723+0000","last_fullsized":"2023-12-17T12:08:50.166723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:16:55.476924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.165976+0000","last_change":"2023-12-17T12:08:49.202082+0000","last_active":"2023-12-17T12:08:50.165976+0000","last_peered":"2023-12-17T12:08:50.165976+0000","last_clean":"2023-12-17T12:08:50.165976+0000","last_became_active":"2023-12-17T12:08:49.200583+0000","last_became_peered":"2023-12-17T12:08:49.200583+0000","last_unstale":"2023-12-17T12:08:50.165976+0000","last_undegraded":"2023-12-17T12:08:50.165976+0000","last_fullsized":"2023-12-17T12:08:50.165976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:19:25.112916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799351+0000","last_change":"2023-12-17T12:08:49.184442+0000","last_active":"2023-12-17T12:08:50.799351+0000","last_peered":"2023-12-17T12:08:50.799351+0000","last_clean":"2023-12-17T12:08:50.799351+0000","last_became_active":"2023-12-17T12:08:49.184053+0000","last_became_peered":"2023-12-17T12:08:49.184053+0000","last_unstale":"2023-12-17T12:08:50.799351+0000","last_undegraded":"2023-12-17T12:08:50.799351+0000","last_fullsized":"2023-12-17T12:08:50.799351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:38:18.864989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799166+0000","last_change":"2023-12-17T12:08:49.185550+0000","last_active":"2023-12-17T12:08:50.799166+0000","last_peered":"2023-12-17T12:08:50.799166+0000","last_clean":"2023-12-17T12:08:50.799166+0000","last_became_active":"2023-12-17T12:08:49.185190+0000","last_became_peered":"2023-12-17T12:08:49.185190+0000","last_unstale":"2023-12-17T12:08:50.799166+0000","last_undegraded":"2023-12-17T12:08:50.799166+0000","last_fullsized":"2023-12-17T12:08:50.799166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:11:29.821074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800305+0000","last_change":"2023-12-17T12:08:49.182738+0000","last_active":"2023-12-17T12:08:50.800305+0000","last_peered":"2023-12-17T12:08:50.800305+0000","last_clean":"2023-12-17T12:08:50.800305+0000","last_became_active":"2023-12-17T12:08:49.182451+0000","last_became_peered":"2023-12-17T12:08:49.182451+0000","last_unstale":"2023-12-17T12:08:50.800305+0000","last_undegraded":"2023-12-17T12:08:50.800305+0000","last_fullsized":"2023-12-17T12:08:50.800305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:21:06.788880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801010+0000","last_change":"2023-12-17T12:08:49.182062+0000","last_active":"2023-12-17T12:08:50.801010+0000","last_peered":"2023-12-17T12:08:50.801010+0000","last_clean":"2023-12-17T12:08:50.801010+0000","last_became_active":"2023-12-17T12:08:49.181725+0000","last_became_peered":"2023-12-17T12:08:49.181725+0000","last_unstale":"2023-12-17T12:08:50.801010+0000","last_undegraded":"2023-12-17T12:08:50.801010+0000","last_fullsized":"2023-12-17T12:08:50.801010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:06:26.626520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801161+0000","last_change":"2023-12-17T12:08:49.174849+0000","last_active":"2023-12-17T12:08:50.801161+0000","last_peered":"2023-12-17T12:08:50.801161+0000","last_clean":"2023-12-17T12:08:50.801161+0000","last_became_active":"2023-12-17T12:08:49.174408+0000","last_became_peered":"2023-12-17T12:08:49.174408+0000","last_unstale":"2023-12-17T12:08:50.801161+0000","last_undegraded":"2023-12-17T12:08:50.801161+0000","last_fullsized":"2023-12-17T12:08:50.801161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:36:11.050556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800708+0000","last_change":"2023-12-17T12:08:49.202639+0000","last_active":"2023-12-17T12:08:50.800708+0000","last_peered":"2023-12-17T12:08:50.800708+0000","last_clean":"2023-12-17T12:08:50.800708+0000","last_became_active":"2023-12-17T12:08:49.202400+0000","last_became_peered":"2023-12-17T12:08:49.202400+0000","last_unstale":"2023-12-17T12:08:50.800708+0000","last_undegraded":"2023-12-17T12:08:50.800708+0000","last_fullsized":"2023-12-17T12:08:50.800708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:45:27.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.429695+0000","last_change":"2023-12-17T12:08:48.400771+0000","last_active":"2023-12-17T12:08:54.429695+0000","last_peered":"2023-12-17T12:08:54.429695+0000","last_clean":"2023-12-17T12:08:54.429695+0000","last_became_active":"2023-12-17T12:08:48.400430+0000","last_became_peered":"2023-12-17T12:08:48.400430+0000","last_unstale":"2023-12-17T12:08:54.429695+0000","last_undegraded":"2023-12-17T12:08:54.429695+0000","last_fullsized":"2023-12-17T12:08:54.429695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:58:32.453329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800500+0000","last_change":"2023-12-17T12:08:49.196221+0000","last_active":"2023-12-17T12:08:50.800500+0000","last_peered":"2023-12-17T12:08:50.800500+0000","last_clean":"2023-12-17T12:08:50.800500+0000","last_became_active":"2023-12-17T12:08:49.195908+0000","last_became_peered":"2023-12-17T12:08:49.195908+0000","last_unstale":"2023-12-17T12:08:50.800500+0000","last_undegraded":"2023-12-17T12:08:50.800500+0000","last_fullsized":"2023-12-17T12:08:50.800500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:28:17.710818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.890182+0000","last_change":"2023-12-17T12:08:48.408069+0000","last_active":"2023-12-17T12:08:49.890182+0000","last_peered":"2023-12-17T12:08:49.890182+0000","last_clean":"2023-12-17T12:08:49.890182+0000","last_became_active":"2023-12-17T12:08:48.407209+0000","last_became_peered":"2023-12-17T12:08:48.407209+0000","last_unstale":"2023-12-17T12:08:49.890182+0000","last_undegraded":"2023-12-17T12:08:49.890182+0000","last_fullsized":"2023-12-17T12:08:49.890182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:24:08.191642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800626+0000","last_change":"2023-12-17T12:08:49.179982+0000","last_active":"2023-12-17T12:08:50.800626+0000","last_peered":"2023-12-17T12:08:50.800626+0000","last_clean":"2023-12-17T12:08:50.800626+0000","last_became_active":"2023-12-17T12:08:49.179610+0000","last_became_peered":"2023-12-17T12:08:49.179610+0000","last_unstale":"2023-12-17T12:08:50.800626+0000","last_undegraded":"2023-12-17T12:08:50.800626+0000","last_fullsized":"2023-12-17T12:08:50.800626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:39:13.133693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.426785+0000","last_change":"2023-12-17T12:08:48.395609+0000","last_active":"2023-12-17T12:08:54.426785+0000","last_peered":"2023-12-17T12:08:54.426785+0000","last_clean":"2023-12-17T12:08:54.426785+0000","last_became_active":"2023-12-17T12:08:48.393809+0000","last_became_peered":"2023-12-17T12:08:48.393809+0000","last_unstale":"2023-12-17T12:08:54.426785+0000","last_undegraded":"2023-12-17T12:08:54.426785+0000","last_fullsized":"2023-12-17T12:08:54.426785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:45:39.698185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799951+0000","last_change":"2023-12-17T12:08:49.182636+0000","last_active":"2023-12-17T12:08:50.799951+0000","last_peered":"2023-12-17T12:08:50.799951+0000","last_clean":"2023-12-17T12:08:50.799951+0000","last_became_active":"2023-12-17T12:08:49.182328+0000","last_became_peered":"2023-12-17T12:08:49.182328+0000","last_unstale":"2023-12-17T12:08:50.799951+0000","last_undegraded":"2023-12-17T12:08:50.799951+0000","last_fullsized":"2023-12-17T12:08:50.799951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:19:27.674053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.892433+0000","last_change":"2023-12-17T12:08:48.405333+0000","last_active":"2023-12-17T12:08:49.892433+0000","last_peered":"2023-12-17T12:08:49.892433+0000","last_clean":"2023-12-17T12:08:49.892433+0000","last_became_active":"2023-12-17T12:08:48.403558+0000","last_became_peered":"2023-12-17T12:08:48.403558+0000","last_unstale":"2023-12-17T12:08:49.892433+0000","last_undegraded":"2023-12-17T12:08:49.892433+0000","last_fullsized":"2023-12-17T12:08:49.892433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:16:48.657674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801556+0000","last_change":"2023-12-17T12:08:49.187999+0000","last_active":"2023-12-17T12:08:50.801556+0000","last_peered":"2023-12-17T12:08:50.801556+0000","last_clean":"2023-12-17T12:08:50.801556+0000","last_became_active":"2023-12-17T12:08:49.187698+0000","last_became_peered":"2023-12-17T12:08:49.187698+0000","last_unstale":"2023-12-17T12:08:50.801556+0000","last_undegraded":"2023-12-17T12:08:50.801556+0000","last_fullsized":"2023-12-17T12:08:50.801556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:52:22.279227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:57.438266+0000","last_change":"2023-12-17T12:08:48.403205+0000","last_active":"2023-12-17T12:08:57.438266+0000","last_peered":"2023-12-17T12:08:57.438266+0000","last_clean":"2023-12-17T12:08:57.438266+0000","last_became_active":"2023-12-17T12:08:48.401930+0000","last_became_peered":"2023-12-17T12:08:48.401930+0000","last_unstale":"2023-12-17T12:08:57.438266+0000","last_undegraded":"2023-12-17T12:08:57.438266+0000","last_fullsized":"2023-12-17T12:08:57.438266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:28:03.889769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166873+0000","last_change":"2023-12-17T12:08:49.176597+0000","last_active":"2023-12-17T12:08:50.166873+0000","last_peered":"2023-12-17T12:08:50.166873+0000","last_clean":"2023-12-17T12:08:50.166873+0000","last_became_active":"2023-12-17T12:08:49.176279+0000","last_became_peered":"2023-12-17T12:08:49.176279+0000","last_unstale":"2023-12-17T12:08:50.166873+0000","last_undegraded":"2023-12-17T12:08:50.166873+0000","last_fullsized":"2023-12-17T12:08:50.166873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:38:48.558555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.891293+0000","last_change":"2023-12-17T12:08:48.390768+0000","last_active":"2023-12-17T12:08:49.891293+0000","last_peered":"2023-12-17T12:08:49.891293+0000","last_clean":"2023-12-17T12:08:49.891293+0000","last_became_active":"2023-12-17T12:08:48.390452+0000","last_became_peered":"2023-12-17T12:08:48.390452+0000","last_unstale":"2023-12-17T12:08:49.891293+0000","last_undegraded":"2023-12-17T12:08:49.891293+0000","last_fullsized":"2023-12-17T12:08:49.891293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:16:50.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801298+0000","last_change":"2023-12-17T12:08:49.194554+0000","last_active":"2023-12-17T12:08:50.801298+0000","last_peered":"2023-12-17T12:08:50.801298+0000","last_clean":"2023-12-17T12:08:50.801298+0000","last_became_active":"2023-12-17T12:08:49.193794+0000","last_became_peered":"2023-12-17T12:08:49.193794+0000","last_unstale":"2023-12-17T12:08:50.801298+0000","last_undegraded":"2023-12-17T12:08:50.801298+0000","last_fullsized":"2023-12-17T12:08:50.801298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:19:30.140621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.428699+0000","last_change":"2023-12-17T12:08:48.388949+0000","last_active":"2023-12-17T12:08:54.428699+0000","last_peered":"2023-12-17T12:08:54.428699+0000","last_clean":"2023-12-17T12:08:54.428699+0000","last_became_active":"2023-12-17T12:08:48.386904+0000","last_became_peered":"2023-12-17T12:08:48.386904+0000","last_unstale":"2023-12-17T12:08:54.428699+0000","last_undegraded":"2023-12-17T12:08:54.428699+0000","last_fullsized":"2023-12-17T12:08:54.428699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:25:42.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799743+0000","last_change":"2023-12-17T12:08:49.197069+0000","last_active":"2023-12-17T12:08:50.799743+0000","last_peered":"2023-12-17T12:08:50.799743+0000","last_clean":"2023-12-17T12:08:50.799743+0000","last_became_active":"2023-12-17T12:08:49.196366+0000","last_became_peered":"2023-12-17T12:08:49.196366+0000","last_unstale":"2023-12-17T12:08:50.799743+0000","last_undegraded":"2023-12-17T12:08:50.799743+0000","last_fullsized":"2023-12-17T12:08:50.799743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:43:41.262255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800127+0000","last_change":"2023-12-17T12:08:48.401890+0000","last_active":"2023-12-17T12:08:50.800127+0000","last_peered":"2023-12-17T12:08:50.800127+0000","last_clean":"2023-12-17T12:08:50.800127+0000","last_became_active":"2023-12-17T12:08:48.396456+0000","last_became_peered":"2023-12-17T12:08:48.396456+0000","last_unstale":"2023-12-17T12:08:50.800127+0000","last_undegraded":"2023-12-17T12:08:50.800127+0000","last_fullsized":"2023-12-17T12:08:50.800127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:15:10.389287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800048+0000","last_change":"2023-12-17T12:08:49.192837+0000","last_active":"2023-12-17T12:08:50.800048+0000","last_peered":"2023-12-17T12:08:50.800048+0000","last_clean":"2023-12-17T12:08:50.800048+0000","last_became_active":"2023-12-17T12:08:49.192479+0000","last_became_peered":"2023-12-17T12:08:49.192479+0000","last_unstale":"2023-12-17T12:08:50.800048+0000","last_undegraded":"2023-12-17T12:08:50.800048+0000","last_fullsized":"2023-12-17T12:08:50.800048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:08:50.771280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.421363+0000","last_change":"2023-12-17T12:08:48.395661+0000","last_active":"2023-12-17T12:08:53.421363+0000","last_peered":"2023-12-17T12:08:53.421363+0000","last_clean":"2023-12-17T12:08:53.421363+0000","last_became_active":"2023-12-17T12:08:48.395282+0000","last_became_peered":"2023-12-17T12:08:48.395282+0000","last_unstale":"2023-12-17T12:08:53.421363+0000","last_undegraded":"2023-12-17T12:08:53.421363+0000","last_fullsized":"2023-12-17T12:08:53.421363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:02:15.554520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799977+0000","last_change":"2023-12-17T12:08:49.179339+0000","last_active":"2023-12-17T12:08:50.799977+0000","last_peered":"2023-12-17T12:08:50.799977+0000","last_clean":"2023-12-17T12:08:50.799977+0000","last_became_active":"2023-12-17T12:08:49.179041+0000","last_became_peered":"2023-12-17T12:08:49.179041+0000","last_unstale":"2023-12-17T12:08:50.799977+0000","last_undegraded":"2023-12-17T12:08:50.799977+0000","last_fullsized":"2023-12-17T12:08:50.799977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:27:28.508576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.416798+0000","last_change":"2023-12-17T12:08:48.401911+0000","last_active":"2023-12-17T12:08:53.416798+0000","last_peered":"2023-12-17T12:08:53.416798+0000","last_clean":"2023-12-17T12:08:53.416798+0000","last_became_active":"2023-12-17T12:08:48.401552+0000","last_became_peered":"2023-12-17T12:08:48.401552+0000","last_unstale":"2023-12-17T12:08:53.416798+0000","last_undegraded":"2023-12-17T12:08:53.416798+0000","last_fullsized":"2023-12-17T12:08:53.416798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:40:54.813575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800494+0000","last_change":"2023-12-17T12:08:49.177683+0000","last_active":"2023-12-17T12:08:50.800494+0000","last_peered":"2023-12-17T12:08:50.800494+0000","last_clean":"2023-12-17T12:08:50.800494+0000","last_became_active":"2023-12-17T12:08:49.177313+0000","last_became_peered":"2023-12-17T12:08:49.177313+0000","last_unstale":"2023-12-17T12:08:50.800494+0000","last_undegraded":"2023-12-17T12:08:50.800494+0000","last_fullsized":"2023-12-17T12:08:50.800494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:29:40.397954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.417886+0000","last_change":"2023-12-17T12:08:48.392907+0000","last_active":"2023-12-17T12:08:53.417886+0000","last_peered":"2023-12-17T12:08:53.417886+0000","last_clean":"2023-12-17T12:08:53.417886+0000","last_became_active":"2023-12-17T12:08:48.392504+0000","last_became_peered":"2023-12-17T12:08:48.392504+0000","last_unstale":"2023-12-17T12:08:53.417886+0000","last_undegraded":"2023-12-17T12:08:53.417886+0000","last_fullsized":"2023-12-17T12:08:53.417886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:25:24.561271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800436+0000","last_change":"2023-12-17T12:08:49.186687+0000","last_active":"2023-12-17T12:08:50.800436+0000","last_peered":"2023-12-17T12:08:50.800436+0000","last_clean":"2023-12-17T12:08:50.800436+0000","last_became_active":"2023-12-17T12:08:49.186400+0000","last_became_peered":"2023-12-17T12:08:49.186400+0000","last_unstale":"2023-12-17T12:08:50.800436+0000","last_undegraded":"2023-12-17T12:08:50.800436+0000","last_fullsized":"2023-12-17T12:08:50.800436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:12:27.143793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.421156+0000","last_change":"2023-12-17T12:08:48.401743+0000","last_active":"2023-12-17T12:08:53.421156+0000","last_peered":"2023-12-17T12:08:53.421156+0000","last_clean":"2023-12-17T12:08:53.421156+0000","last_became_active":"2023-12-17T12:08:48.401369+0000","last_became_peered":"2023-12-17T12:08:48.401369+0000","last_unstale":"2023-12-17T12:08:53.421156+0000","last_undegraded":"2023-12-17T12:08:53.421156+0000","last_fullsized":"2023-12-17T12:08:53.421156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:40:39.725727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.167071+0000","last_change":"2023-12-17T12:08:49.176410+0000","last_active":"2023-12-17T12:08:50.167071+0000","last_peered":"2023-12-17T12:08:50.167071+0000","last_clean":"2023-12-17T12:08:50.167071+0000","last_became_active":"2023-12-17T12:08:49.176120+0000","last_became_peered":"2023-12-17T12:08:49.176120+0000","last_unstale":"2023-12-17T12:08:50.167071+0000","last_undegraded":"2023-12-17T12:08:50.167071+0000","last_fullsized":"2023-12-17T12:08:50.167071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:28:18.241319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.427999+0000","last_change":"2023-12-17T12:08:48.405781+0000","last_active":"2023-12-17T12:08:54.427999+0000","last_peered":"2023-12-17T12:08:54.427999+0000","last_clean":"2023-12-17T12:08:54.427999+0000","last_became_active":"2023-12-17T12:08:48.405255+0000","last_became_peered":"2023-12-17T12:08:48.405255+0000","last_unstale":"2023-12-17T12:08:54.427999+0000","last_undegraded":"2023-12-17T12:08:54.427999+0000","last_fullsized":"2023-12-17T12:08:54.427999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:00:10.046901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.165717+0000","last_change":"2023-12-17T12:08:49.199457+0000","last_active":"2023-12-17T12:08:50.165717+0000","last_peered":"2023-12-17T12:08:50.165717+0000","last_clean":"2023-12-17T12:08:50.165717+0000","last_became_active":"2023-12-17T12:08:49.197147+0000","last_became_peered":"2023-12-17T12:08:49.197147+0000","last_unstale":"2023-12-17T12:08:50.165717+0000","last_undegraded":"2023-12-17T12:08:50.165717+0000","last_fullsized":"2023-12-17T12:08:50.165717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:36:46.619674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.417790+0000","last_change":"2023-12-17T12:08:48.407127+0000","last_active":"2023-12-17T12:08:53.417790+0000","last_peered":"2023-12-17T12:08:53.417790+0000","last_clean":"2023-12-17T12:08:53.417790+0000","last_became_active":"2023-12-17T12:08:48.406726+0000","last_became_peered":"2023-12-17T12:08:48.406726+0000","last_unstale":"2023-12-17T12:08:53.417790+0000","last_undegraded":"2023-12-17T12:08:53.417790+0000","last_fullsized":"2023-12-17T12:08:53.417790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:16:19.492428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800682+0000","last_change":"2023-12-17T12:08:49.188478+0000","last_active":"2023-12-17T12:08:50.800682+0000","last_peered":"2023-12-17T12:08:50.800682+0000","last_clean":"2023-12-17T12:08:50.800682+0000","last_became_active":"2023-12-17T12:08:49.188161+0000","last_became_peered":"2023-12-17T12:08:49.188161+0000","last_unstale":"2023-12-17T12:08:50.800682+0000","last_undegraded":"2023-12-17T12:08:50.800682+0000","last_fullsized":"2023-12-17T12:08:50.800682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:02:14.484656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:55.434379+0000","last_change":"2023-12-17T12:08:48.393092+0000","last_active":"2023-12-17T12:08:55.434379+0000","last_peered":"2023-12-17T12:08:55.434379+0000","last_clean":"2023-12-17T12:08:55.434379+0000","last_became_active":"2023-12-17T12:08:48.392702+0000","last_became_peered":"2023-12-17T12:08:48.392702+0000","last_unstale":"2023-12-17T12:08:55.434379+0000","last_undegraded":"2023-12-17T12:08:55.434379+0000","last_fullsized":"2023-12-17T12:08:55.434379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:53:00.261870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":977,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800064+0000","last_change":"2023-12-17T12:08:49.181367+0000","last_active":"2023-12-17T12:08:50.800064+0000","last_peered":"2023-12-17T12:08:50.800064+0000","last_clean":"2023-12-17T12:08:50.800064+0000","last_became_active":"2023-12-17T12:08:49.180841+0000","last_became_peered":"2023-12-17T12:08:49.180841+0000","last_unstale":"2023-12-17T12:08:50.800064+0000","last_undegraded":"2023-12-17T12:08:50.800064+0000","last_fullsized":"2023-12-17T12:08:50.800064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:57:33.920366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.430590+0000","last_change":"2023-12-17T12:08:48.398863+0000","last_active":"2023-12-17T12:08:54.430590+0000","last_peered":"2023-12-17T12:08:54.430590+0000","last_clean":"2023-12-17T12:08:54.430590+0000","last_became_active":"2023-12-17T12:08:48.396178+0000","last_became_peered":"2023-12-17T12:08:48.396178+0000","last_unstale":"2023-12-17T12:08:54.430590+0000","last_undegraded":"2023-12-17T12:08:54.430590+0000","last_fullsized":"2023-12-17T12:08:54.430590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:31:44.593246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800598+0000","last_change":"2023-12-17T12:08:49.190795+0000","last_active":"2023-12-17T12:08:50.800598+0000","last_peered":"2023-12-17T12:08:50.800598+0000","last_clean":"2023-12-17T12:08:50.800598+0000","last_became_active":"2023-12-17T12:08:49.190468+0000","last_became_peered":"2023-12-17T12:08:49.190468+0000","last_unstale":"2023-12-17T12:08:50.800598+0000","last_undegraded":"2023-12-17T12:08:50.800598+0000","last_fullsized":"2023-12-17T12:08:50.800598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:26:37.302852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.891883+0000","last_change":"2023-12-17T12:08:48.391524+0000","last_active":"2023-12-17T12:08:49.891883+0000","last_peered":"2023-12-17T12:08:49.891883+0000","last_clean":"2023-12-17T12:08:49.891883+0000","last_became_active":"2023-12-17T12:08:48.391195+0000","last_became_peered":"2023-12-17T12:08:48.391195+0000","last_unstale":"2023-12-17T12:08:49.891883+0000","last_undegraded":"2023-12-17T12:08:49.891883+0000","last_fullsized":"2023-12-17T12:08:49.891883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:33:55.399168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800953+0000","last_change":"2023-12-17T12:08:49.188376+0000","last_active":"2023-12-17T12:08:50.800953+0000","last_peered":"2023-12-17T12:08:50.800953+0000","last_clean":"2023-12-17T12:08:50.800953+0000","last_became_active":"2023-12-17T12:08:49.188038+0000","last_became_peered":"2023-12-17T12:08:49.188038+0000","last_unstale":"2023-12-17T12:08:50.800953+0000","last_undegraded":"2023-12-17T12:08:50.800953+0000","last_fullsized":"2023-12-17T12:08:50.800953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:27:35.263053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.164120+0000","last_change":"2023-12-17T12:08:48.401451+0000","last_active":"2023-12-17T12:08:49.164120+0000","last_peered":"2023-12-17T12:08:49.164120+0000","last_clean":"2023-12-17T12:08:49.164120+0000","last_became_active":"2023-12-17T12:08:48.401100+0000","last_became_peered":"2023-12-17T12:08:48.401100+0000","last_unstale":"2023-12-17T12:08:49.164120+0000","last_undegraded":"2023-12-17T12:08:49.164120+0000","last_fullsized":"2023-12-17T12:08:49.164120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:38:48.372997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801722+0000","last_change":"2023-12-17T12:08:49.194033+0000","last_active":"2023-12-17T12:08:50.801722+0000","last_peered":"2023-12-17T12:08:50.801722+0000","last_clean":"2023-12-17T12:08:50.801722+0000","last_became_active":"2023-12-17T12:08:49.192344+0000","last_became_peered":"2023-12-17T12:08:49.192344+0000","last_unstale":"2023-12-17T12:08:50.801722+0000","last_undegraded":"2023-12-17T12:08:50.801722+0000","last_fullsized":"2023-12-17T12:08:50.801722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:00:40.106655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800354+0000","last_change":"2023-12-17T12:08:48.397060+0000","last_active":"2023-12-17T12:08:50.800354+0000","last_peered":"2023-12-17T12:08:50.800354+0000","last_clean":"2023-12-17T12:08:50.800354+0000","last_became_active":"2023-12-17T12:08:48.396704+0000","last_became_peered":"2023-12-17T12:08:48.396704+0000","last_unstale":"2023-12-17T12:08:50.800354+0000","last_undegraded":"2023-12-17T12:08:50.800354+0000","last_fullsized":"2023-12-17T12:08:50.800354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:31:08.882089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801631+0000","last_change":"2023-12-17T12:08:49.183915+0000","last_active":"2023-12-17T12:08:50.801631+0000","last_peered":"2023-12-17T12:08:50.801631+0000","last_clean":"2023-12-17T12:08:50.801631+0000","last_became_active":"2023-12-17T12:08:49.183615+0000","last_became_peered":"2023-12-17T12:08:49.183615+0000","last_unstale":"2023-12-17T12:08:50.801631+0000","last_undegraded":"2023-12-17T12:08:50.801631+0000","last_fullsized":"2023-12-17T12:08:50.801631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:29:09.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800390+0000","last_change":"2023-12-17T12:08:49.190952+0000","last_active":"2023-12-17T12:08:50.800390+0000","last_peered":"2023-12-17T12:08:50.800390+0000","last_clean":"2023-12-17T12:08:50.800390+0000","last_became_active":"2023-12-17T12:08:49.190586+0000","last_became_peered":"2023-12-17T12:08:49.190586+0000","last_unstale":"2023-12-17T12:08:50.800390+0000","last_undegraded":"2023-12-17T12:08:50.800390+0000","last_fullsized":"2023-12-17T12:08:50.800390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:44:00.972429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.430383+0000","last_change":"2023-12-17T12:08:48.409488+0000","last_active":"2023-12-17T12:08:54.430383+0000","last_peered":"2023-12-17T12:08:54.430383+0000","last_clean":"2023-12-17T12:08:54.430383+0000","last_became_active":"2023-12-17T12:08:48.408703+0000","last_became_peered":"2023-12-17T12:08:48.408703+0000","last_unstale":"2023-12-17T12:08:54.430383+0000","last_undegraded":"2023-12-17T12:08:54.430383+0000","last_fullsized":"2023-12-17T12:08:54.430383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:33:14.861530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800861+0000","last_change":"2023-12-17T12:08:49.197960+0000","last_active":"2023-12-17T12:08:50.800861+0000","last_peered":"2023-12-17T12:08:50.800861+0000","last_clean":"2023-12-17T12:08:50.800861+0000","last_became_active":"2023-12-17T12:08:49.196484+0000","last_became_peered":"2023-12-17T12:08:49.196484+0000","last_unstale":"2023-12-17T12:08:50.800861+0000","last_undegraded":"2023-12-17T12:08:50.800861+0000","last_fullsized":"2023-12-17T12:08:50.800861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:53:50.293026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:55.427998+0000","last_change":"2023-12-17T12:08:48.387589+0000","last_active":"2023-12-17T12:08:55.427998+0000","last_peered":"2023-12-17T12:08:55.427998+0000","last_clean":"2023-12-17T12:08:55.427998+0000","last_became_active":"2023-12-17T12:08:48.387258+0000","last_became_peered":"2023-12-17T12:08:48.387258+0000","last_unstale":"2023-12-17T12:08:55.427998+0000","last_undegraded":"2023-12-17T12:08:55.427998+0000","last_fullsized":"2023-12-17T12:08:55.427998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:29:30.815005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":318,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.167785+0000","last_change":"2023-12-17T12:07:42.894663+0000","last_active":"2023-12-17T12:08:49.167785+0000","last_peered":"2023-12-17T12:08:49.167785+0000","last_clean":"2023-12-17T12:08:49.167785+0000","last_became_active":"2023-12-17T12:07:42.894199+0000","last_became_peered":"2023-12-17T12:07:42.894199+0000","last_unstale":"2023-12-17T12:08:49.167785+0000","last_undegraded":"2023-12-17T12:08:49.167785+0000","last_fullsized":"2023-12-17T12:08:49.167785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:04:41.531220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:04:41.531220+0000","last_clean_scrub_stamp":"2023-12-17T12:04:41.531220+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:07:49.358197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.165667+0000","last_change":"2023-12-17T12:08:49.195358+0000","last_active":"2023-12-17T12:08:50.165667+0000","last_peered":"2023-12-17T12:08:50.165667+0000","last_clean":"2023-12-17T12:08:50.165667+0000","last_became_active":"2023-12-17T12:08:49.194996+0000","last_became_peered":"2023-12-17T12:08:49.194996+0000","last_unstale":"2023-12-17T12:08:50.165667+0000","last_undegraded":"2023-12-17T12:08:50.165667+0000","last_fullsized":"2023-12-17T12:08:50.165667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:23:10.016755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.429330+0000","last_change":"2023-12-17T12:08:48.394937+0000","last_active":"2023-12-17T12:08:54.429330+0000","last_peered":"2023-12-17T12:08:54.429330+0000","last_clean":"2023-12-17T12:08:54.429330+0000","last_became_active":"2023-12-17T12:08:48.394590+0000","last_became_peered":"2023-12-17T12:08:48.394590+0000","last_unstale":"2023-12-17T12:08:54.429330+0000","last_undegraded":"2023-12-17T12:08:54.429330+0000","last_fullsized":"2023-12-17T12:08:54.429330+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:24:49.088883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.167099+0000","last_change":"2023-12-17T12:08:48.396769+0000","last_active":"2023-12-17T12:08:50.167099+0000","last_peered":"2023-12-17T12:08:50.167099+0000","last_clean":"2023-12-17T12:08:50.167099+0000","last_became_active":"2023-12-17T12:08:48.395984+0000","last_became_peered":"2023-12-17T12:08:48.395984+0000","last_unstale":"2023-12-17T12:08:50.167099+0000","last_undegraded":"2023-12-17T12:08:50.167099+0000","last_fullsized":"2023-12-17T12:08:50.167099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:34:59.872771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.165878+0000","last_change":"2023-12-17T12:08:49.194598+0000","last_active":"2023-12-17T12:08:50.165878+0000","last_peered":"2023-12-17T12:08:50.165878+0000","last_clean":"2023-12-17T12:08:50.165878+0000","last_became_active":"2023-12-17T12:08:49.189450+0000","last_became_peered":"2023-12-17T12:08:49.189450+0000","last_unstale":"2023-12-17T12:08:50.165878+0000","last_undegraded":"2023-12-17T12:08:50.165878+0000","last_fullsized":"2023-12-17T12:08:50.165878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:45:57.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.428399+0000","last_change":"2023-12-17T12:08:48.407902+0000","last_active":"2023-12-17T12:08:54.428399+0000","last_peered":"2023-12-17T12:08:54.428399+0000","last_clean":"2023-12-17T12:08:54.428399+0000","last_became_active":"2023-12-17T12:08:48.407601+0000","last_became_peered":"2023-12-17T12:08:48.407601+0000","last_unstale":"2023-12-17T12:08:54.428399+0000","last_undegraded":"2023-12-17T12:08:54.428399+0000","last_fullsized":"2023-12-17T12:08:54.428399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:56:34.626900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800934+0000","last_change":"2023-12-17T12:08:49.196489+0000","last_active":"2023-12-17T12:08:50.800934+0000","last_peered":"2023-12-17T12:08:50.800934+0000","last_clean":"2023-12-17T12:08:50.800934+0000","last_became_active":"2023-12-17T12:08:49.196063+0000","last_became_peered":"2023-12-17T12:08:49.196063+0000","last_unstale":"2023-12-17T12:08:50.800934+0000","last_undegraded":"2023-12-17T12:08:50.800934+0000","last_fullsized":"2023-12-17T12:08:50.800934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:16:08.658987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.421014+0000","last_change":"2023-12-17T12:08:48.400075+0000","last_active":"2023-12-17T12:08:53.421014+0000","last_peered":"2023-12-17T12:08:53.421014+0000","last_clean":"2023-12-17T12:08:53.421014+0000","last_became_active":"2023-12-17T12:08:48.398728+0000","last_became_peered":"2023-12-17T12:08:48.398728+0000","last_unstale":"2023-12-17T12:08:53.421014+0000","last_undegraded":"2023-12-17T12:08:53.421014+0000","last_fullsized":"2023-12-17T12:08:53.421014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:10:21.065038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799938+0000","last_change":"2023-12-17T12:08:49.193200+0000","last_active":"2023-12-17T12:08:50.799938+0000","last_peered":"2023-12-17T12:08:50.799938+0000","last_clean":"2023-12-17T12:08:50.799938+0000","last_became_active":"2023-12-17T12:08:49.192855+0000","last_became_peered":"2023-12-17T12:08:49.192855+0000","last_unstale":"2023-12-17T12:08:50.799938+0000","last_undegraded":"2023-12-17T12:08:50.799938+0000","last_fullsized":"2023-12-17T12:08:50.799938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:14:09.089956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.418408+0000","last_change":"2023-12-17T12:08:48.404661+0000","last_active":"2023-12-17T12:08:53.418408+0000","last_peered":"2023-12-17T12:08:53.418408+0000","last_clean":"2023-12-17T12:08:53.418408+0000","last_became_active":"2023-12-17T12:08:48.404336+0000","last_became_peered":"2023-12-17T12:08:48.404336+0000","last_unstale":"2023-12-17T12:08:53.418408+0000","last_undegraded":"2023-12-17T12:08:53.418408+0000","last_fullsized":"2023-12-17T12:08:53.418408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:37:20.863966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166701+0000","last_change":"2023-12-17T12:08:49.181169+0000","last_active":"2023-12-17T12:08:50.166701+0000","last_peered":"2023-12-17T12:08:50.166701+0000","last_clean":"2023-12-17T12:08:50.166701+0000","last_became_active":"2023-12-17T12:08:49.177802+0000","last_became_peered":"2023-12-17T12:08:49.177802+0000","last_unstale":"2023-12-17T12:08:50.166701+0000","last_undegraded":"2023-12-17T12:08:50.166701+0000","last_fullsized":"2023-12-17T12:08:50.166701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:56:50.122246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.417528+0000","last_change":"2023-12-17T12:08:48.407865+0000","last_active":"2023-12-17T12:08:53.417528+0000","last_peered":"2023-12-17T12:08:53.417528+0000","last_clean":"2023-12-17T12:08:53.417528+0000","last_became_active":"2023-12-17T12:08:48.407580+0000","last_became_peered":"2023-12-17T12:08:48.407580+0000","last_unstale":"2023-12-17T12:08:53.417528+0000","last_undegraded":"2023-12-17T12:08:53.417528+0000","last_fullsized":"2023-12-17T12:08:53.417528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:10:20.505630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800530+0000","last_change":"2023-12-17T12:08:49.192724+0000","last_active":"2023-12-17T12:08:50.800530+0000","last_peered":"2023-12-17T12:08:50.800530+0000","last_clean":"2023-12-17T12:08:50.800530+0000","last_became_active":"2023-12-17T12:08:49.188658+0000","last_became_peered":"2023-12-17T12:08:49.188658+0000","last_unstale":"2023-12-17T12:08:50.800530+0000","last_undegraded":"2023-12-17T12:08:50.800530+0000","last_fullsized":"2023-12-17T12:08:50.800530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:00:17.493003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:51.314265+0000","last_change":"2023-12-17T12:08:48.404717+0000","last_active":"2023-12-17T12:08:51.314265+0000","last_peered":"2023-12-17T12:08:51.314265+0000","last_clean":"2023-12-17T12:08:51.314265+0000","last_became_active":"2023-12-17T12:08:48.404426+0000","last_became_peered":"2023-12-17T12:08:48.404426+0000","last_unstale":"2023-12-17T12:08:51.314265+0000","last_undegraded":"2023-12-17T12:08:51.314265+0000","last_fullsized":"2023-12-17T12:08:51.314265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:35:28.090855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800627+0000","last_change":"2023-12-17T12:08:49.191911+0000","last_active":"2023-12-17T12:08:50.800627+0000","last_peered":"2023-12-17T12:08:50.800627+0000","last_clean":"2023-12-17T12:08:50.800627+0000","last_became_active":"2023-12-17T12:08:49.191573+0000","last_became_peered":"2023-12-17T12:08:49.191573+0000","last_unstale":"2023-12-17T12:08:50.800627+0000","last_undegraded":"2023-12-17T12:08:50.800627+0000","last_fullsized":"2023-12-17T12:08:50.800627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:39:35.354221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.421954+0000","last_change":"2023-12-17T12:08:48.401253+0000","last_active":"2023-12-17T12:08:53.421954+0000","last_peered":"2023-12-17T12:08:53.421954+0000","last_clean":"2023-12-17T12:08:53.421954+0000","last_became_active":"2023-12-17T12:08:48.399675+0000","last_became_peered":"2023-12-17T12:08:48.399675+0000","last_unstale":"2023-12-17T12:08:53.421954+0000","last_undegraded":"2023-12-17T12:08:53.421954+0000","last_fullsized":"2023-12-17T12:08:53.421954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:15:47.969526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800959+0000","last_change":"2023-12-17T12:08:49.191906+0000","last_active":"2023-12-17T12:08:50.800959+0000","last_peered":"2023-12-17T12:08:50.800959+0000","last_clean":"2023-12-17T12:08:50.800959+0000","last_became_active":"2023-12-17T12:08:49.191572+0000","last_became_peered":"2023-12-17T12:08:49.191572+0000","last_unstale":"2023-12-17T12:08:50.800959+0000","last_undegraded":"2023-12-17T12:08:50.800959+0000","last_fullsized":"2023-12-17T12:08:50.800959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:56:05.670034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.430335+0000","last_change":"2023-12-17T12:08:48.399004+0000","last_active":"2023-12-17T12:08:54.430335+0000","last_peered":"2023-12-17T12:08:54.430335+0000","last_clean":"2023-12-17T12:08:54.430335+0000","last_became_active":"2023-12-17T12:08:48.398453+0000","last_became_peered":"2023-12-17T12:08:48.398453+0000","last_unstale":"2023-12-17T12:08:54.430335+0000","last_undegraded":"2023-12-17T12:08:54.430335+0000","last_fullsized":"2023-12-17T12:08:54.430335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:30:53.767008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800987+0000","last_change":"2023-12-17T12:08:49.178859+0000","last_active":"2023-12-17T12:08:50.800987+0000","last_peered":"2023-12-17T12:08:50.800987+0000","last_clean":"2023-12-17T12:08:50.800987+0000","last_became_active":"2023-12-17T12:08:49.178119+0000","last_became_peered":"2023-12-17T12:08:49.178119+0000","last_unstale":"2023-12-17T12:08:50.800987+0000","last_undegraded":"2023-12-17T12:08:50.800987+0000","last_fullsized":"2023-12-17T12:08:50.800987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:32:40.758242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801506+0000","last_change":"2023-12-17T12:08:48.407249+0000","last_active":"2023-12-17T12:08:50.801506+0000","last_peered":"2023-12-17T12:08:50.801506+0000","last_clean":"2023-12-17T12:08:50.801506+0000","last_became_active":"2023-12-17T12:08:48.406572+0000","last_became_peered":"2023-12-17T12:08:48.406572+0000","last_unstale":"2023-12-17T12:08:50.801506+0000","last_undegraded":"2023-12-17T12:08:50.801506+0000","last_fullsized":"2023-12-17T12:08:50.801506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:19:07.751233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:52.426195+0000","last_change":"2023-12-17T12:08:49.192590+0000","last_active":"2023-12-17T12:08:52.426195+0000","last_peered":"2023-12-17T12:08:52.426195+0000","last_clean":"2023-12-17T12:08:52.426195+0000","last_became_active":"2023-12-17T12:08:49.192250+0000","last_became_peered":"2023-12-17T12:08:49.192250+0000","last_unstale":"2023-12-17T12:08:52.426195+0000","last_undegraded":"2023-12-17T12:08:52.426195+0000","last_fullsized":"2023-12-17T12:08:52.426195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:51:20.599412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166058+0000","last_change":"2023-12-17T12:08:48.406799+0000","last_active":"2023-12-17T12:08:50.166058+0000","last_peered":"2023-12-17T12:08:50.166058+0000","last_clean":"2023-12-17T12:08:50.166058+0000","last_became_active":"2023-12-17T12:08:48.406423+0000","last_became_peered":"2023-12-17T12:08:48.406423+0000","last_unstale":"2023-12-17T12:08:50.166058+0000","last_undegraded":"2023-12-17T12:08:50.166058+0000","last_fullsized":"2023-12-17T12:08:50.166058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:43:55.600399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801260+0000","last_change":"2023-12-17T12:08:49.182840+0000","last_active":"2023-12-17T12:08:50.801260+0000","last_peered":"2023-12-17T12:08:50.801260+0000","last_clean":"2023-12-17T12:08:50.801260+0000","last_became_active":"2023-12-17T12:08:49.181562+0000","last_became_peered":"2023-12-17T12:08:49.181562+0000","last_unstale":"2023-12-17T12:08:50.801260+0000","last_undegraded":"2023-12-17T12:08:50.801260+0000","last_fullsized":"2023-12-17T12:08:50.801260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:01:41.059044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800478+0000","last_change":"2023-12-17T12:08:49.193449+0000","last_active":"2023-12-17T12:08:50.800478+0000","last_peered":"2023-12-17T12:08:50.800478+0000","last_clean":"2023-12-17T12:08:50.800478+0000","last_became_active":"2023-12-17T12:08:49.192942+0000","last_became_peered":"2023-12-17T12:08:49.192942+0000","last_unstale":"2023-12-17T12:08:50.800478+0000","last_undegraded":"2023-12-17T12:08:50.800478+0000","last_fullsized":"2023-12-17T12:08:50.800478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:42:51.021808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799882+0000","last_change":"2023-12-17T12:08:49.193515+0000","last_active":"2023-12-17T12:08:50.799882+0000","last_peered":"2023-12-17T12:08:50.799882+0000","last_clean":"2023-12-17T12:08:50.799882+0000","last_became_active":"2023-12-17T12:08:49.193232+0000","last_became_peered":"2023-12-17T12:08:49.193232+0000","last_unstale":"2023-12-17T12:08:50.799882+0000","last_undegraded":"2023-12-17T12:08:50.799882+0000","last_fullsized":"2023-12-17T12:08:50.799882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:26:59.948350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799983+0000","last_change":"2023-12-17T12:08:49.197110+0000","last_active":"2023-12-17T12:08:50.799983+0000","last_peered":"2023-12-17T12:08:50.799983+0000","last_clean":"2023-12-17T12:08:50.799983+0000","last_became_active":"2023-12-17T12:08:49.196712+0000","last_became_peered":"2023-12-17T12:08:49.196712+0000","last_unstale":"2023-12-17T12:08:50.799983+0000","last_undegraded":"2023-12-17T12:08:50.799983+0000","last_fullsized":"2023-12-17T12:08:50.799983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:21:40.555247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799414+0000","last_change":"2023-12-17T12:08:49.192021+0000","last_active":"2023-12-17T12:08:50.799414+0000","last_peered":"2023-12-17T12:08:50.799414+0000","last_clean":"2023-12-17T12:08:50.799414+0000","last_became_active":"2023-12-17T12:08:49.191696+0000","last_became_peered":"2023-12-17T12:08:49.191696+0000","last_unstale":"2023-12-17T12:08:50.799414+0000","last_undegraded":"2023-12-17T12:08:50.799414+0000","last_fullsized":"2023-12-17T12:08:50.799414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:27:40.425617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166136+0000","last_change":"2023-12-17T12:08:49.196208+0000","last_active":"2023-12-17T12:08:50.166136+0000","last_peered":"2023-12-17T12:08:50.166136+0000","last_clean":"2023-12-17T12:08:50.166136+0000","last_became_active":"2023-12-17T12:08:49.195900+0000","last_became_peered":"2023-12-17T12:08:49.195900+0000","last_unstale":"2023-12-17T12:08:50.166136+0000","last_undegraded":"2023-12-17T12:08:50.166136+0000","last_fullsized":"2023-12-17T12:08:50.166136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:07:05.201192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800886+0000","last_change":"2023-12-17T12:08:49.191147+0000","last_active":"2023-12-17T12:08:50.800886+0000","last_peered":"2023-12-17T12:08:50.800886+0000","last_clean":"2023-12-17T12:08:50.800886+0000","last_became_active":"2023-12-17T12:08:49.190804+0000","last_became_peered":"2023-12-17T12:08:49.190804+0000","last_unstale":"2023-12-17T12:08:50.800886+0000","last_undegraded":"2023-12-17T12:08:50.800886+0000","last_fullsized":"2023-12-17T12:08:50.800886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:09:40.551342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801114+0000","last_change":"2023-12-17T12:08:49.184707+0000","last_active":"2023-12-17T12:08:50.801114+0000","last_peered":"2023-12-17T12:08:50.801114+0000","last_clean":"2023-12-17T12:08:50.801114+0000","last_became_active":"2023-12-17T12:08:49.183644+0000","last_became_peered":"2023-12-17T12:08:49.183644+0000","last_unstale":"2023-12-17T12:08:50.801114+0000","last_undegraded":"2023-12-17T12:08:50.801114+0000","last_fullsized":"2023-12-17T12:08:50.801114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:30:43.845054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800791+0000","last_change":"2023-12-17T12:08:49.195299+0000","last_active":"2023-12-17T12:08:50.800791+0000","last_peered":"2023-12-17T12:08:50.800791+0000","last_clean":"2023-12-17T12:08:50.800791+0000","last_became_active":"2023-12-17T12:08:49.194315+0000","last_became_peered":"2023-12-17T12:08:49.194315+0000","last_unstale":"2023-12-17T12:08:50.800791+0000","last_undegraded":"2023-12-17T12:08:50.800791+0000","last_fullsized":"2023-12-17T12:08:50.800791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:33:26.056073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800236+0000","last_change":"2023-12-17T12:08:49.196145+0000","last_active":"2023-12-17T12:08:50.800236+0000","last_peered":"2023-12-17T12:08:50.800236+0000","last_clean":"2023-12-17T12:08:50.800236+0000","last_became_active":"2023-12-17T12:08:49.195893+0000","last_became_peered":"2023-12-17T12:08:49.195893+0000","last_unstale":"2023-12-17T12:08:50.800236+0000","last_undegraded":"2023-12-17T12:08:50.800236+0000","last_fullsized":"2023-12-17T12:08:50.800236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:33:23.868301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800225+0000","last_change":"2023-12-17T12:08:49.193113+0000","last_active":"2023-12-17T12:08:50.800225+0000","last_peered":"2023-12-17T12:08:50.800225+0000","last_clean":"2023-12-17T12:08:50.800225+0000","last_became_active":"2023-12-17T12:08:49.191730+0000","last_became_peered":"2023-12-17T12:08:49.191730+0000","last_unstale":"2023-12-17T12:08:50.800225+0000","last_undegraded":"2023-12-17T12:08:50.800225+0000","last_fullsized":"2023-12-17T12:08:50.800225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:45:33.858040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800050+0000","last_change":"2023-12-17T12:08:49.193401+0000","last_active":"2023-12-17T12:08:50.800050+0000","last_peered":"2023-12-17T12:08:50.800050+0000","last_clean":"2023-12-17T12:08:50.800050+0000","last_became_active":"2023-12-17T12:08:49.192993+0000","last_became_peered":"2023-12-17T12:08:49.192993+0000","last_unstale":"2023-12-17T12:08:50.800050+0000","last_undegraded":"2023-12-17T12:08:50.800050+0000","last_fullsized":"2023-12-17T12:08:50.800050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:21:16.594494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166016+0000","last_change":"2023-12-17T12:08:49.200282+0000","last_active":"2023-12-17T12:08:50.166016+0000","last_peered":"2023-12-17T12:08:50.166016+0000","last_clean":"2023-12-17T12:08:50.166016+0000","last_became_active":"2023-12-17T12:08:49.199788+0000","last_became_peered":"2023-12-17T12:08:49.199788+0000","last_unstale":"2023-12-17T12:08:50.166016+0000","last_undegraded":"2023-12-17T12:08:50.166016+0000","last_fullsized":"2023-12-17T12:08:50.166016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:07:19.164767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800199+0000","last_change":"2023-12-17T12:08:49.185712+0000","last_active":"2023-12-17T12:08:50.800199+0000","last_peered":"2023-12-17T12:08:50.800199+0000","last_clean":"2023-12-17T12:08:50.800199+0000","last_became_active":"2023-12-17T12:08:49.185417+0000","last_became_peered":"2023-12-17T12:08:49.185417+0000","last_unstale":"2023-12-17T12:08:50.800199+0000","last_undegraded":"2023-12-17T12:08:50.800199+0000","last_fullsized":"2023-12-17T12:08:50.800199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:28:43.076574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246978,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5611,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":122,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":33096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481805,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1017058,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54800000000000004}]},{"osd":2,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.72499999999999998}]},{"osd":3,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77000000000000002}]},{"osd":4,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.58099999999999996}]},{"osd":5,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.89000000000000001}]},{"osd":7,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.82599999999999996}]},{"osd":8,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.81999999999999995}]},{"osd":9,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.86899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]}]},{"osd":9,"up_from":64,"seq":274877906965,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":424940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.442},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.85099999999999998}]},{"osd":2,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":13.166,"5min":13.166,"15min":13.166},"last":0.58799999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.66500000000000004}]},{"osd":4,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.75}]},{"osd":5,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":7.1920000000000002,"5min":7.1920000000000002,"15min":7.1920000000000002},"last":0.621},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.64200000000000002}]},{"osd":6,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.88800000000000001}]},{"osd":8,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":13.18,"5min":13.18,"15min":13.18},"last":0.78700000000000003}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.87},{"interface":"front","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":13.212,"5min":13.212,"15min":13.212},"last":0.93400000000000005}]}]},{"osd":8,"up_from":56,"seq":240518168602,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":424748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":7.0350000000000001,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":7.0119999999999996,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":1.135}]},{"osd":2,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.90300000000000002}]},{"osd":3,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":7.0599999999999996,"5min":7.0599999999999996,"15min":7.0599999999999996},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":6.992,"5min":6.992,"15min":6.992},"last":1.097}]},{"osd":4,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":1.179},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.2549999999999999}]},{"osd":5,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.232}]},{"osd":6,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.47},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.276}]},{"osd":7,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.331},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.4470000000000001}]},{"osd":9,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.4910000000000001}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.5149999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.631}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397535,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1016463,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92200000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57599999999999996}]},{"osd":1,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.496,"15min":0.48099999999999998},"min":{"1min":0.38900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.94699999999999995}]},{"osd":2,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.78900000000000003,"15min":0.745},"min":{"1min":0.52800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":7.1070000000000002,"5min":7.1070000000000002,"15min":7.1070000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.96899999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95899999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78300000000000003}]},{"osd":4,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94999999999999996,"5min":1.119,"15min":1.119},"last":0.73999999999999999}]},{"osd":5,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.84899999999999998,"15min":0.80100000000000005},"min":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":7.1669999999999998,"5min":7.1669999999999998,"15min":7.1669999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.069}]},{"osd":6,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.02},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.84199999999999997,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0780000000000001,"5min":1.21,"15min":1.21},"last":0.995}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.69799999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.143,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.125},"min":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.101,"5min":7.1369999999999996,"15min":7.1369999999999996},"last":1.0880000000000001}]},{"osd":9,"last update":"Sun Dec 17 12:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.048},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.167}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.1930000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]}]},{"osd":6,"up_from":44,"seq":188978561060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":424212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.67100000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68000000000000005}]},{"osd":1,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.72799999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.77600000000000002,"15min":0.746},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":5.6740000000000004,"5min":5.6740000000000004,"15min":5.6740000000000004},"last":0.59699999999999998}]},{"osd":3,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.45500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.80200000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0529999999999999,"5min":1.167,"15min":1.167},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.85799999999999998,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.72899999999999998}]},{"osd":7,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.91500000000000004}]},{"osd":8,"last update":"Sun Dec 17 12:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.755}]},{"osd":9,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.93700000000000006}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.61699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":425734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.58599999999999997,"15min":0.56100000000000005},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.62,"15min":0.58599999999999997},"min":{"1min":0.53800000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.626}]},{"osd":2,"last update":"Sun Dec 17 12:08:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63800000000000001,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0549999999999999,"5min":1.111,"15min":1.111},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63400000000000001,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.097,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1}]},{"osd":3,"last update":"Sun Dec 17 12:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.623,"15min":0.59199999999999997},"min":{"1min":0.53900000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.622,"15min":0.58999999999999997},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Sun Dec 17 12:08:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79200000000000004}]},{"osd":5,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.69699999999999995,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.70199999999999996}]},{"osd":6,"last update":"Sun Dec 17 12:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.59299999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.751,"15min":0.73199999999999998},"min":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.84099999999999997}]},{"osd":7,"last update":"Sun Dec 17 12:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68899999999999995,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81599999999999995}]},{"osd":8,"last update":"Sun Dec 17 12:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":1.054},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0649999999999999}]},{"osd":9,"last update":"Sun Dec 17 12:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.018}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.92300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":425600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Dec 17 12:08:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.63100000000000001,"15min":0.61099999999999999},"min":{"1min":0.51400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.157},"last":0.504},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.66200000000000003,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Sun Dec 17 12:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65900000000000003,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65000000000000002,"15min":0.59999999999999998},"min":{"1min":0.55900000000000005,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Dec 17 12:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.375,"5min":0.315,"15min":0.315},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79600000000000004}]},{"osd":4,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.64500000000000002},"min":{"1min":0.504,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.077,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.58299999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.171,"5min":1.248,"15min":1.248},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Dec 17 12:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0329999999999999,"5min":1.083,"15min":1.083},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.51800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.83399999999999996},"min":{"1min":0.71999999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.871,"15min":0.86299999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.81599999999999995}]},{"osd":8,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.91100000000000003}]},{"osd":9,"last update":"Sun Dec 17 12:08:20 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.82799999999999996}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.98499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.52400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.48199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.58699999999999997}]},{"osd":1,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.66800000000000004,"15min":0.621},"min":{"1min":0.54500000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98099999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Dec 17 12:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.627}]},{"osd":4,"last update":"Sun Dec 17 12:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.69199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95799999999999996,"5min":1.181,"15min":1.181},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94699999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.82399999999999995}]},{"osd":6,"last update":"Sun Dec 17 12:08:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.73199999999999998,"15min":0.70699999999999996},"min":{"1min":0.67800000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92300000000000004,"5min":1.214,"15min":1.214},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.75900000000000001}]},{"osd":8,"last update":"Sun Dec 17 12:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88600000000000001}]},{"osd":9,"last update":"Sun Dec 17 12:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.80500000000000005}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.96299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":424316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41599999999999998,"15min":0.39200000000000002},"min":{"1min":0.34399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.44700000000000001,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.51600000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.63800000000000001,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.54600000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80300000000000005}]},{"osd":2,"last update":"Sun Dec 17 12:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.69499999999999995,"15min":0.67600000000000005},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.69199999999999995,"15min":0.66100000000000003},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.86099999999999999}]},{"osd":4,"last update":"Sun Dec 17 12:08:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.67500000000000004,"15min":0.63500000000000001},"min":{"1min":0.623,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.66400000000000003}]},{"osd":5,"last update":"Sun Dec 17 12:07:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.876},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75,"15min":0.72799999999999998},"min":{"1min":0.52700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Dec 17 12:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.46400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.754},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.51500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84999999999999998,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90300000000000002}]},{"osd":8,"last update":"Sun Dec 17 12:07:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.71999999999999997}]},{"osd":9,"last update":"Sun Dec 17 12:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.95099999999999996}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.006}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953516,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":424766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.49299999999999999,"15min":0.46700000000000003},"min":{"1min":0.36399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.45900000000000002,"15min":0.435},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.77600000000000002,"15min":0.82599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.087,"5min":5.125,"15min":5.125},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.64300000000000002,"15min":0.61599999999999999},"min":{"1min":0.46000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65700000000000003,"15min":0.629},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Dec 17 12:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68400000000000005,"15min":0.65900000000000003},"min":{"1min":0.59599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.874},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68799999999999994,"15min":0.66800000000000004},"min":{"1min":0.57599999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90600000000000003,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.70999999999999996}]},{"osd":6,"last update":"Sun Dec 17 12:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.93899999999999995,"15min":0.96799999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.389,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.754,"15min":0.746},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.63200000000000001}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.70299999999999996,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.81799999999999995}]},{"osd":8,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.74199999999999999}]},{"osd":9,"last update":"Sun Dec 17 12:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.91200000000000003}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.97999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":5,"up_from":37,"seq":158913789993,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":1016089,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94999999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.64200000000000002}]},{"osd":1,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.55100000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89400000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.752},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66200000000000003,"15min":0.63500000000000001},"min":{"1min":0.53400000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.876,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.39700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95099999999999996,"5min":1.097,"15min":1.097},"last":0.623}]},{"osd":4,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":0.92700000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.52600000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95499999999999996,"5min":1.048,"15min":1.048},"last":0.81200000000000006}]},{"osd":6,"last update":"Sun Dec 17 12:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.62,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.94799999999999995}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":1.0589999999999999,"5min":1.117,"15min":1.117},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":0.83899999999999997},"min":{"1min":0.63500000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.95899999999999996,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.105,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.875},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.83999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1419999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.79200000000000004}]},{"osd":9,"last update":"Sun Dec 17 12:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.96699999999999997}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":1.01},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.91000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34109,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34109,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-17T12:08:58.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 -- 172.21.15.27:0/3571457983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 msgr2=0x7f4134106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 --2- 172.21.15.27:0/3571457983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 0x7f4134106c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4124009990 tx=0x7f4124009ca0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 -- 172.21.15.27:0/3571457983 shutdown_connections 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 --2- 172.21.15.27:0/3571457983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4134101f00 0x7f41341042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 --2- 172.21.15.27:0/3571457983 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4134107160 0x7f41341095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 --2- 172.21.15.27:0/3571457983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 0x7f4134106c20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 -- 172.21.15.27:0/3571457983 >> 172.21.15.27:0/3571457983 conn(0x7f41340fb8f0 msgr2=0x7f41340fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 -- 172.21.15.27:0/3571457983 shutdown_connections 2023-12-17T12:08:59.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.194+0000 7f413bdff700 1 -- 172.21.15.27:0/3571457983 wait complete. 2023-12-17T12:08:59.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.195+0000 7f413bdff700 1 Processor -- start 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.195+0000 7f413bdff700 1 -- start start 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413bdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4134101f00 0x7f4134101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413bdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 0x7f41340ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413bdff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4134107160 0x7f41341003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413bdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4134100b90 con 0x7f4134104830 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413bdff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4134100d00 con 0x7f4134107160 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413bdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41341a13b0 con 0x7f4134101f00 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413939a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 0x7f41340ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f4139b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4134101f00 0x7f4134101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413939a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 0x7f41340ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59366/0 (socket says 172.21.15.27:59366) 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413939a700 1 -- 172.21.15.27:0/1324264411 learned_addr learned my addr 172.21.15.27:0/1324264411 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413a39c700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4134107160 0x7f41341003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413939a700 1 -- 172.21.15.27:0/1324264411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4134101f00 msgr2=0x7f4134101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:59.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413939a700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4134101f00 0x7f4134101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.196+0000 7f413939a700 1 -- 172.21.15.27:0/1324264411 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4134107160 msgr2=0x7f41341003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f413939a700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4134107160 0x7f41341003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f413939a700 1 -- 172.21.15.27:0/1324264411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41240095f0 con 0x7f4134104830 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f4139b9b700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4134101f00 0x7f4134101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f413a39c700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4134107160 0x7f41341003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f413939a700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 0x7f41340ffeb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4124006010 tx=0x7f4124004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f412affd700 1 -- 172.21.15.27:0/1324264411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412401d070 con 0x7f4134104830 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f412affd700 1 -- 172.21.15.27:0/1324264411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f412400bdb0 con 0x7f4134104830 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41341a15e0 con 0x7f4134104830 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f412affd700 1 -- 172.21.15.27:0/1324264411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412400fdf0 con 0x7f4134104830 2023-12-17T12:08:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.197+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41341a1a00 con 0x7f4134104830 2023-12-17T12:08:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.198+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41340fd470 con 0x7f4134104830 2023-12-17T12:08:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.200+0000 7f412affd700 1 -- 172.21.15.27:0/1324264411 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f412400b920 con 0x7f4134104830 2023-12-17T12:08:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.201+0000 7f412affd700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41200a1590 0x7f41200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:08:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.202+0000 7f412affd700 1 -- 172.21.15.27:0/1324264411 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f41240ce130 con 0x7f4134104830 2023-12-17T12:08:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.202+0000 7f4139b9b700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41200a1590 0x7f41200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:08:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.202+0000 7f4139b9b700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41200a1590 0x7f41200a3a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4130011490 tx=0x7f4130009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:08:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.205+0000 7f412affd700 1 -- 172.21.15.27:0/1324264411 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4124097770 con 0x7f4134104830 2023-12-17T12:08:59.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.415+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f413402d3d0 con 0x7f41200a1590 2023-12-17T12:08:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.425+0000 7f412affd700 1 -- 172.21.15.27:0/1324264411 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334461 (secure 0 0 0) 0x7f413402d3d0 con 0x7f41200a1590 2023-12-17T12:08:59.431 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:08:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41200a1590 msgr2=0x7f41200a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41200a1590 0x7f41200a3a50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4130011490 tx=0x7f4130009260 comp rx=0 tx=0).stop 2023-12-17T12:08:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 msgr2=0x7f41340ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:08:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 0x7f41340ffeb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4124006010 tx=0x7f4124004730 comp rx=0 tx=0).stop 2023-12-17T12:08:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 shutdown_connections 2023-12-17T12:08:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f41200a1590 0x7f41200a3a50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4134101f00 0x7f4134101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4134107160 0x7f41341003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 --2- 172.21.15.27:0/1324264411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4134104830 0x7f41340ffeb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:08:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 >> 172.21.15.27:0/1324264411 conn(0x7f41340fb8f0 msgr2=0x7f4134103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:08:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 shutdown_connections 2023-12-17T12:08:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:08:59.430+0000 7f413bdff700 1 -- 172.21.15.27:0/1324264411 wait complete. 2023-12-17T12:08:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-17T12:08:59.451 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2023-12-17T12:08:58.796718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596102,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":155,"num_read_kb":133,"num_write":304,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334896,"kb_used_data":10704,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606928,"statfs":{"total":1151940427776,"available":1151597494272,"internally_reserved":0,"allocated":10960896,"data_stored":6893078,"data_compressed":107256,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19139,"internal_metadata":331855165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5734,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":123,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.634281"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801249+0000","last_change":"2023-12-17T12:08:49.184999+0000","last_active":"2023-12-17T12:08:50.801249+0000","last_peered":"2023-12-17T12:08:50.801249+0000","last_clean":"2023-12-17T12:08:50.801249+0000","last_became_active":"2023-12-17T12:08:49.184214+0000","last_became_peered":"2023-12-17T12:08:49.184214+0000","last_unstale":"2023-12-17T12:08:50.801249+0000","last_undegraded":"2023-12-17T12:08:50.801249+0000","last_fullsized":"2023-12-17T12:08:50.801249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:48:56.174610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799801+0000","last_change":"2023-12-17T12:08:49.188212+0000","last_active":"2023-12-17T12:08:50.799801+0000","last_peered":"2023-12-17T12:08:50.799801+0000","last_clean":"2023-12-17T12:08:50.799801+0000","last_became_active":"2023-12-17T12:08:49.187869+0000","last_became_peered":"2023-12-17T12:08:49.187869+0000","last_unstale":"2023-12-17T12:08:50.799801+0000","last_undegraded":"2023-12-17T12:08:50.799801+0000","last_fullsized":"2023-12-17T12:08:50.799801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:53:52.485169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799658+0000","last_change":"2023-12-17T12:08:49.182801+0000","last_active":"2023-12-17T12:08:50.799658+0000","last_peered":"2023-12-17T12:08:50.799658+0000","last_clean":"2023-12-17T12:08:50.799658+0000","last_became_active":"2023-12-17T12:08:49.182488+0000","last_became_peered":"2023-12-17T12:08:49.182488+0000","last_unstale":"2023-12-17T12:08:50.799658+0000","last_undegraded":"2023-12-17T12:08:50.799658+0000","last_fullsized":"2023-12-17T12:08:50.799658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:16:43.566881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166805+0000","last_change":"2023-12-17T12:08:49.189279+0000","last_active":"2023-12-17T12:08:50.166805+0000","last_peered":"2023-12-17T12:08:50.166805+0000","last_clean":"2023-12-17T12:08:50.166805+0000","last_became_active":"2023-12-17T12:08:49.186419+0000","last_became_peered":"2023-12-17T12:08:49.186419+0000","last_unstale":"2023-12-17T12:08:50.166805+0000","last_undegraded":"2023-12-17T12:08:50.166805+0000","last_fullsized":"2023-12-17T12:08:50.166805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:47:03.035599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166781+0000","last_change":"2023-12-17T12:08:49.192587+0000","last_active":"2023-12-17T12:08:50.166781+0000","last_peered":"2023-12-17T12:08:50.166781+0000","last_clean":"2023-12-17T12:08:50.166781+0000","last_became_active":"2023-12-17T12:08:49.192272+0000","last_became_peered":"2023-12-17T12:08:49.192272+0000","last_unstale":"2023-12-17T12:08:50.166781+0000","last_undegraded":"2023-12-17T12:08:50.166781+0000","last_fullsized":"2023-12-17T12:08:50.166781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:45:30.131065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801258+0000","last_change":"2023-12-17T12:08:49.176906+0000","last_active":"2023-12-17T12:08:50.801258+0000","last_peered":"2023-12-17T12:08:50.801258+0000","last_clean":"2023-12-17T12:08:50.801258+0000","last_became_active":"2023-12-17T12:08:49.176591+0000","last_became_peered":"2023-12-17T12:08:49.176591+0000","last_unstale":"2023-12-17T12:08:50.801258+0000","last_undegraded":"2023-12-17T12:08:50.801258+0000","last_fullsized":"2023-12-17T12:08:50.801258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:27:17.508628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800023+0000","last_change":"2023-12-17T12:08:49.177409+0000","last_active":"2023-12-17T12:08:50.800023+0000","last_peered":"2023-12-17T12:08:50.800023+0000","last_clean":"2023-12-17T12:08:50.800023+0000","last_became_active":"2023-12-17T12:08:49.177093+0000","last_became_peered":"2023-12-17T12:08:49.177093+0000","last_unstale":"2023-12-17T12:08:50.800023+0000","last_undegraded":"2023-12-17T12:08:50.800023+0000","last_fullsized":"2023-12-17T12:08:50.800023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:56:30.669406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799992+0000","last_change":"2023-12-17T12:08:49.194035+0000","last_active":"2023-12-17T12:08:50.799992+0000","last_peered":"2023-12-17T12:08:50.799992+0000","last_clean":"2023-12-17T12:08:50.799992+0000","last_became_active":"2023-12-17T12:08:49.191677+0000","last_became_peered":"2023-12-17T12:08:49.191677+0000","last_unstale":"2023-12-17T12:08:50.799992+0000","last_undegraded":"2023-12-17T12:08:50.799992+0000","last_fullsized":"2023-12-17T12:08:50.799992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:21:00.984977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800536+0000","last_change":"2023-12-17T12:08:49.187140+0000","last_active":"2023-12-17T12:08:50.800536+0000","last_peered":"2023-12-17T12:08:50.800536+0000","last_clean":"2023-12-17T12:08:50.800536+0000","last_became_active":"2023-12-17T12:08:49.186807+0000","last_became_peered":"2023-12-17T12:08:49.186807+0000","last_unstale":"2023-12-17T12:08:50.800536+0000","last_undegraded":"2023-12-17T12:08:50.800536+0000","last_fullsized":"2023-12-17T12:08:50.800536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:28:59.060986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801544+0000","last_change":"2023-12-17T12:08:49.186371+0000","last_active":"2023-12-17T12:08:50.801544+0000","last_peered":"2023-12-17T12:08:50.801544+0000","last_clean":"2023-12-17T12:08:50.801544+0000","last_became_active":"2023-12-17T12:08:49.186045+0000","last_became_peered":"2023-12-17T12:08:49.186045+0000","last_unstale":"2023-12-17T12:08:50.801544+0000","last_undegraded":"2023-12-17T12:08:50.801544+0000","last_fullsized":"2023-12-17T12:08:50.801544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:56:47.851617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800846+0000","last_change":"2023-12-17T12:08:49.179088+0000","last_active":"2023-12-17T12:08:50.800846+0000","last_peered":"2023-12-17T12:08:50.800846+0000","last_clean":"2023-12-17T12:08:50.800846+0000","last_became_active":"2023-12-17T12:08:49.178545+0000","last_became_peered":"2023-12-17T12:08:49.178545+0000","last_unstale":"2023-12-17T12:08:50.800846+0000","last_undegraded":"2023-12-17T12:08:50.800846+0000","last_fullsized":"2023-12-17T12:08:50.800846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:52:58.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166723+0000","last_change":"2023-12-17T12:08:49.187681+0000","last_active":"2023-12-17T12:08:50.166723+0000","last_peered":"2023-12-17T12:08:50.166723+0000","last_clean":"2023-12-17T12:08:50.166723+0000","last_became_active":"2023-12-17T12:08:49.187086+0000","last_became_peered":"2023-12-17T12:08:49.187086+0000","last_unstale":"2023-12-17T12:08:50.166723+0000","last_undegraded":"2023-12-17T12:08:50.166723+0000","last_fullsized":"2023-12-17T12:08:50.166723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:16:55.476924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.165976+0000","last_change":"2023-12-17T12:08:49.202082+0000","last_active":"2023-12-17T12:08:50.165976+0000","last_peered":"2023-12-17T12:08:50.165976+0000","last_clean":"2023-12-17T12:08:50.165976+0000","last_became_active":"2023-12-17T12:08:49.200583+0000","last_became_peered":"2023-12-17T12:08:49.200583+0000","last_unstale":"2023-12-17T12:08:50.165976+0000","last_undegraded":"2023-12-17T12:08:50.165976+0000","last_fullsized":"2023-12-17T12:08:50.165976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:19:25.112916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799351+0000","last_change":"2023-12-17T12:08:49.184442+0000","last_active":"2023-12-17T12:08:50.799351+0000","last_peered":"2023-12-17T12:08:50.799351+0000","last_clean":"2023-12-17T12:08:50.799351+0000","last_became_active":"2023-12-17T12:08:49.184053+0000","last_became_peered":"2023-12-17T12:08:49.184053+0000","last_unstale":"2023-12-17T12:08:50.799351+0000","last_undegraded":"2023-12-17T12:08:50.799351+0000","last_fullsized":"2023-12-17T12:08:50.799351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:38:18.864989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799166+0000","last_change":"2023-12-17T12:08:49.185550+0000","last_active":"2023-12-17T12:08:50.799166+0000","last_peered":"2023-12-17T12:08:50.799166+0000","last_clean":"2023-12-17T12:08:50.799166+0000","last_became_active":"2023-12-17T12:08:49.185190+0000","last_became_peered":"2023-12-17T12:08:49.185190+0000","last_unstale":"2023-12-17T12:08:50.799166+0000","last_undegraded":"2023-12-17T12:08:50.799166+0000","last_fullsized":"2023-12-17T12:08:50.799166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:11:29.821074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800305+0000","last_change":"2023-12-17T12:08:49.182738+0000","last_active":"2023-12-17T12:08:50.800305+0000","last_peered":"2023-12-17T12:08:50.800305+0000","last_clean":"2023-12-17T12:08:50.800305+0000","last_became_active":"2023-12-17T12:08:49.182451+0000","last_became_peered":"2023-12-17T12:08:49.182451+0000","last_unstale":"2023-12-17T12:08:50.800305+0000","last_undegraded":"2023-12-17T12:08:50.800305+0000","last_fullsized":"2023-12-17T12:08:50.800305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:21:06.788880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801010+0000","last_change":"2023-12-17T12:08:49.182062+0000","last_active":"2023-12-17T12:08:50.801010+0000","last_peered":"2023-12-17T12:08:50.801010+0000","last_clean":"2023-12-17T12:08:50.801010+0000","last_became_active":"2023-12-17T12:08:49.181725+0000","last_became_peered":"2023-12-17T12:08:49.181725+0000","last_unstale":"2023-12-17T12:08:50.801010+0000","last_undegraded":"2023-12-17T12:08:50.801010+0000","last_fullsized":"2023-12-17T12:08:50.801010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:06:26.626520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801161+0000","last_change":"2023-12-17T12:08:49.174849+0000","last_active":"2023-12-17T12:08:50.801161+0000","last_peered":"2023-12-17T12:08:50.801161+0000","last_clean":"2023-12-17T12:08:50.801161+0000","last_became_active":"2023-12-17T12:08:49.174408+0000","last_became_peered":"2023-12-17T12:08:49.174408+0000","last_unstale":"2023-12-17T12:08:50.801161+0000","last_undegraded":"2023-12-17T12:08:50.801161+0000","last_fullsized":"2023-12-17T12:08:50.801161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:36:11.050556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800708+0000","last_change":"2023-12-17T12:08:49.202639+0000","last_active":"2023-12-17T12:08:50.800708+0000","last_peered":"2023-12-17T12:08:50.800708+0000","last_clean":"2023-12-17T12:08:50.800708+0000","last_became_active":"2023-12-17T12:08:49.202400+0000","last_became_peered":"2023-12-17T12:08:49.202400+0000","last_unstale":"2023-12-17T12:08:50.800708+0000","last_undegraded":"2023-12-17T12:08:50.800708+0000","last_fullsized":"2023-12-17T12:08:50.800708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:45:27.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.429695+0000","last_change":"2023-12-17T12:08:48.400771+0000","last_active":"2023-12-17T12:08:54.429695+0000","last_peered":"2023-12-17T12:08:54.429695+0000","last_clean":"2023-12-17T12:08:54.429695+0000","last_became_active":"2023-12-17T12:08:48.400430+0000","last_became_peered":"2023-12-17T12:08:48.400430+0000","last_unstale":"2023-12-17T12:08:54.429695+0000","last_undegraded":"2023-12-17T12:08:54.429695+0000","last_fullsized":"2023-12-17T12:08:54.429695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:58:32.453329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800500+0000","last_change":"2023-12-17T12:08:49.196221+0000","last_active":"2023-12-17T12:08:50.800500+0000","last_peered":"2023-12-17T12:08:50.800500+0000","last_clean":"2023-12-17T12:08:50.800500+0000","last_became_active":"2023-12-17T12:08:49.195908+0000","last_became_peered":"2023-12-17T12:08:49.195908+0000","last_unstale":"2023-12-17T12:08:50.800500+0000","last_undegraded":"2023-12-17T12:08:50.800500+0000","last_fullsized":"2023-12-17T12:08:50.800500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:28:17.710818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.890182+0000","last_change":"2023-12-17T12:08:48.408069+0000","last_active":"2023-12-17T12:08:49.890182+0000","last_peered":"2023-12-17T12:08:49.890182+0000","last_clean":"2023-12-17T12:08:49.890182+0000","last_became_active":"2023-12-17T12:08:48.407209+0000","last_became_peered":"2023-12-17T12:08:48.407209+0000","last_unstale":"2023-12-17T12:08:49.890182+0000","last_undegraded":"2023-12-17T12:08:49.890182+0000","last_fullsized":"2023-12-17T12:08:49.890182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:24:08.191642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800626+0000","last_change":"2023-12-17T12:08:49.179982+0000","last_active":"2023-12-17T12:08:50.800626+0000","last_peered":"2023-12-17T12:08:50.800626+0000","last_clean":"2023-12-17T12:08:50.800626+0000","last_became_active":"2023-12-17T12:08:49.179610+0000","last_became_peered":"2023-12-17T12:08:49.179610+0000","last_unstale":"2023-12-17T12:08:50.800626+0000","last_undegraded":"2023-12-17T12:08:50.800626+0000","last_fullsized":"2023-12-17T12:08:50.800626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:39:13.133693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.426785+0000","last_change":"2023-12-17T12:08:48.395609+0000","last_active":"2023-12-17T12:08:54.426785+0000","last_peered":"2023-12-17T12:08:54.426785+0000","last_clean":"2023-12-17T12:08:54.426785+0000","last_became_active":"2023-12-17T12:08:48.393809+0000","last_became_peered":"2023-12-17T12:08:48.393809+0000","last_unstale":"2023-12-17T12:08:54.426785+0000","last_undegraded":"2023-12-17T12:08:54.426785+0000","last_fullsized":"2023-12-17T12:08:54.426785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:45:39.698185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799951+0000","last_change":"2023-12-17T12:08:49.182636+0000","last_active":"2023-12-17T12:08:50.799951+0000","last_peered":"2023-12-17T12:08:50.799951+0000","last_clean":"2023-12-17T12:08:50.799951+0000","last_became_active":"2023-12-17T12:08:49.182328+0000","last_became_peered":"2023-12-17T12:08:49.182328+0000","last_unstale":"2023-12-17T12:08:50.799951+0000","last_undegraded":"2023-12-17T12:08:50.799951+0000","last_fullsized":"2023-12-17T12:08:50.799951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:19:27.674053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.892433+0000","last_change":"2023-12-17T12:08:48.405333+0000","last_active":"2023-12-17T12:08:49.892433+0000","last_peered":"2023-12-17T12:08:49.892433+0000","last_clean":"2023-12-17T12:08:49.892433+0000","last_became_active":"2023-12-17T12:08:48.403558+0000","last_became_peered":"2023-12-17T12:08:48.403558+0000","last_unstale":"2023-12-17T12:08:49.892433+0000","last_undegraded":"2023-12-17T12:08:49.892433+0000","last_fullsized":"2023-12-17T12:08:49.892433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:16:48.657674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801556+0000","last_change":"2023-12-17T12:08:49.187999+0000","last_active":"2023-12-17T12:08:50.801556+0000","last_peered":"2023-12-17T12:08:50.801556+0000","last_clean":"2023-12-17T12:08:50.801556+0000","last_became_active":"2023-12-17T12:08:49.187698+0000","last_became_peered":"2023-12-17T12:08:49.187698+0000","last_unstale":"2023-12-17T12:08:50.801556+0000","last_undegraded":"2023-12-17T12:08:50.801556+0000","last_fullsized":"2023-12-17T12:08:50.801556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:52:22.279227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:57.438266+0000","last_change":"2023-12-17T12:08:48.403205+0000","last_active":"2023-12-17T12:08:57.438266+0000","last_peered":"2023-12-17T12:08:57.438266+0000","last_clean":"2023-12-17T12:08:57.438266+0000","last_became_active":"2023-12-17T12:08:48.401930+0000","last_became_peered":"2023-12-17T12:08:48.401930+0000","last_unstale":"2023-12-17T12:08:57.438266+0000","last_undegraded":"2023-12-17T12:08:57.438266+0000","last_fullsized":"2023-12-17T12:08:57.438266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:28:03.889769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166873+0000","last_change":"2023-12-17T12:08:49.176597+0000","last_active":"2023-12-17T12:08:50.166873+0000","last_peered":"2023-12-17T12:08:50.166873+0000","last_clean":"2023-12-17T12:08:50.166873+0000","last_became_active":"2023-12-17T12:08:49.176279+0000","last_became_peered":"2023-12-17T12:08:49.176279+0000","last_unstale":"2023-12-17T12:08:50.166873+0000","last_undegraded":"2023-12-17T12:08:50.166873+0000","last_fullsized":"2023-12-17T12:08:50.166873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:38:48.558555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.891293+0000","last_change":"2023-12-17T12:08:48.390768+0000","last_active":"2023-12-17T12:08:49.891293+0000","last_peered":"2023-12-17T12:08:49.891293+0000","last_clean":"2023-12-17T12:08:49.891293+0000","last_became_active":"2023-12-17T12:08:48.390452+0000","last_became_peered":"2023-12-17T12:08:48.390452+0000","last_unstale":"2023-12-17T12:08:49.891293+0000","last_undegraded":"2023-12-17T12:08:49.891293+0000","last_fullsized":"2023-12-17T12:08:49.891293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:16:50.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801298+0000","last_change":"2023-12-17T12:08:49.194554+0000","last_active":"2023-12-17T12:08:50.801298+0000","last_peered":"2023-12-17T12:08:50.801298+0000","last_clean":"2023-12-17T12:08:50.801298+0000","last_became_active":"2023-12-17T12:08:49.193794+0000","last_became_peered":"2023-12-17T12:08:49.193794+0000","last_unstale":"2023-12-17T12:08:50.801298+0000","last_undegraded":"2023-12-17T12:08:50.801298+0000","last_fullsized":"2023-12-17T12:08:50.801298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:19:30.140621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.428699+0000","last_change":"2023-12-17T12:08:48.388949+0000","last_active":"2023-12-17T12:08:54.428699+0000","last_peered":"2023-12-17T12:08:54.428699+0000","last_clean":"2023-12-17T12:08:54.428699+0000","last_became_active":"2023-12-17T12:08:48.386904+0000","last_became_peered":"2023-12-17T12:08:48.386904+0000","last_unstale":"2023-12-17T12:08:54.428699+0000","last_undegraded":"2023-12-17T12:08:54.428699+0000","last_fullsized":"2023-12-17T12:08:54.428699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:25:42.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799743+0000","last_change":"2023-12-17T12:08:49.197069+0000","last_active":"2023-12-17T12:08:50.799743+0000","last_peered":"2023-12-17T12:08:50.799743+0000","last_clean":"2023-12-17T12:08:50.799743+0000","last_became_active":"2023-12-17T12:08:49.196366+0000","last_became_peered":"2023-12-17T12:08:49.196366+0000","last_unstale":"2023-12-17T12:08:50.799743+0000","last_undegraded":"2023-12-17T12:08:50.799743+0000","last_fullsized":"2023-12-17T12:08:50.799743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:43:41.262255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800127+0000","last_change":"2023-12-17T12:08:48.401890+0000","last_active":"2023-12-17T12:08:50.800127+0000","last_peered":"2023-12-17T12:08:50.800127+0000","last_clean":"2023-12-17T12:08:50.800127+0000","last_became_active":"2023-12-17T12:08:48.396456+0000","last_became_peered":"2023-12-17T12:08:48.396456+0000","last_unstale":"2023-12-17T12:08:50.800127+0000","last_undegraded":"2023-12-17T12:08:50.800127+0000","last_fullsized":"2023-12-17T12:08:50.800127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:15:10.389287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800048+0000","last_change":"2023-12-17T12:08:49.192837+0000","last_active":"2023-12-17T12:08:50.800048+0000","last_peered":"2023-12-17T12:08:50.800048+0000","last_clean":"2023-12-17T12:08:50.800048+0000","last_became_active":"2023-12-17T12:08:49.192479+0000","last_became_peered":"2023-12-17T12:08:49.192479+0000","last_unstale":"2023-12-17T12:08:50.800048+0000","last_undegraded":"2023-12-17T12:08:50.800048+0000","last_fullsized":"2023-12-17T12:08:50.800048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:08:50.771280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.421363+0000","last_change":"2023-12-17T12:08:48.395661+0000","last_active":"2023-12-17T12:08:53.421363+0000","last_peered":"2023-12-17T12:08:53.421363+0000","last_clean":"2023-12-17T12:08:53.421363+0000","last_became_active":"2023-12-17T12:08:48.395282+0000","last_became_peered":"2023-12-17T12:08:48.395282+0000","last_unstale":"2023-12-17T12:08:53.421363+0000","last_undegraded":"2023-12-17T12:08:53.421363+0000","last_fullsized":"2023-12-17T12:08:53.421363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:02:15.554520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799977+0000","last_change":"2023-12-17T12:08:49.179339+0000","last_active":"2023-12-17T12:08:50.799977+0000","last_peered":"2023-12-17T12:08:50.799977+0000","last_clean":"2023-12-17T12:08:50.799977+0000","last_became_active":"2023-12-17T12:08:49.179041+0000","last_became_peered":"2023-12-17T12:08:49.179041+0000","last_unstale":"2023-12-17T12:08:50.799977+0000","last_undegraded":"2023-12-17T12:08:50.799977+0000","last_fullsized":"2023-12-17T12:08:50.799977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:27:28.508576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.416798+0000","last_change":"2023-12-17T12:08:48.401911+0000","last_active":"2023-12-17T12:08:53.416798+0000","last_peered":"2023-12-17T12:08:53.416798+0000","last_clean":"2023-12-17T12:08:53.416798+0000","last_became_active":"2023-12-17T12:08:48.401552+0000","last_became_peered":"2023-12-17T12:08:48.401552+0000","last_unstale":"2023-12-17T12:08:53.416798+0000","last_undegraded":"2023-12-17T12:08:53.416798+0000","last_fullsized":"2023-12-17T12:08:53.416798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:40:54.813575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800494+0000","last_change":"2023-12-17T12:08:49.177683+0000","last_active":"2023-12-17T12:08:50.800494+0000","last_peered":"2023-12-17T12:08:50.800494+0000","last_clean":"2023-12-17T12:08:50.800494+0000","last_became_active":"2023-12-17T12:08:49.177313+0000","last_became_peered":"2023-12-17T12:08:49.177313+0000","last_unstale":"2023-12-17T12:08:50.800494+0000","last_undegraded":"2023-12-17T12:08:50.800494+0000","last_fullsized":"2023-12-17T12:08:50.800494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:29:40.397954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.417886+0000","last_change":"2023-12-17T12:08:48.392907+0000","last_active":"2023-12-17T12:08:53.417886+0000","last_peered":"2023-12-17T12:08:53.417886+0000","last_clean":"2023-12-17T12:08:53.417886+0000","last_became_active":"2023-12-17T12:08:48.392504+0000","last_became_peered":"2023-12-17T12:08:48.392504+0000","last_unstale":"2023-12-17T12:08:53.417886+0000","last_undegraded":"2023-12-17T12:08:53.417886+0000","last_fullsized":"2023-12-17T12:08:53.417886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:25:24.561271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800436+0000","last_change":"2023-12-17T12:08:49.186687+0000","last_active":"2023-12-17T12:08:50.800436+0000","last_peered":"2023-12-17T12:08:50.800436+0000","last_clean":"2023-12-17T12:08:50.800436+0000","last_became_active":"2023-12-17T12:08:49.186400+0000","last_became_peered":"2023-12-17T12:08:49.186400+0000","last_unstale":"2023-12-17T12:08:50.800436+0000","last_undegraded":"2023-12-17T12:08:50.800436+0000","last_fullsized":"2023-12-17T12:08:50.800436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:12:27.143793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.421156+0000","last_change":"2023-12-17T12:08:48.401743+0000","last_active":"2023-12-17T12:08:53.421156+0000","last_peered":"2023-12-17T12:08:53.421156+0000","last_clean":"2023-12-17T12:08:53.421156+0000","last_became_active":"2023-12-17T12:08:48.401369+0000","last_became_peered":"2023-12-17T12:08:48.401369+0000","last_unstale":"2023-12-17T12:08:53.421156+0000","last_undegraded":"2023-12-17T12:08:53.421156+0000","last_fullsized":"2023-12-17T12:08:53.421156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:40:39.725727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.167071+0000","last_change":"2023-12-17T12:08:49.176410+0000","last_active":"2023-12-17T12:08:50.167071+0000","last_peered":"2023-12-17T12:08:50.167071+0000","last_clean":"2023-12-17T12:08:50.167071+0000","last_became_active":"2023-12-17T12:08:49.176120+0000","last_became_peered":"2023-12-17T12:08:49.176120+0000","last_unstale":"2023-12-17T12:08:50.167071+0000","last_undegraded":"2023-12-17T12:08:50.167071+0000","last_fullsized":"2023-12-17T12:08:50.167071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:28:18.241319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.427999+0000","last_change":"2023-12-17T12:08:48.405781+0000","last_active":"2023-12-17T12:08:54.427999+0000","last_peered":"2023-12-17T12:08:54.427999+0000","last_clean":"2023-12-17T12:08:54.427999+0000","last_became_active":"2023-12-17T12:08:48.405255+0000","last_became_peered":"2023-12-17T12:08:48.405255+0000","last_unstale":"2023-12-17T12:08:54.427999+0000","last_undegraded":"2023-12-17T12:08:54.427999+0000","last_fullsized":"2023-12-17T12:08:54.427999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:00:10.046901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.165717+0000","last_change":"2023-12-17T12:08:49.199457+0000","last_active":"2023-12-17T12:08:50.165717+0000","last_peered":"2023-12-17T12:08:50.165717+0000","last_clean":"2023-12-17T12:08:50.165717+0000","last_became_active":"2023-12-17T12:08:49.197147+0000","last_became_peered":"2023-12-17T12:08:49.197147+0000","last_unstale":"2023-12-17T12:08:50.165717+0000","last_undegraded":"2023-12-17T12:08:50.165717+0000","last_fullsized":"2023-12-17T12:08:50.165717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:36:46.619674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.417790+0000","last_change":"2023-12-17T12:08:48.407127+0000","last_active":"2023-12-17T12:08:53.417790+0000","last_peered":"2023-12-17T12:08:53.417790+0000","last_clean":"2023-12-17T12:08:53.417790+0000","last_became_active":"2023-12-17T12:08:48.406726+0000","last_became_peered":"2023-12-17T12:08:48.406726+0000","last_unstale":"2023-12-17T12:08:53.417790+0000","last_undegraded":"2023-12-17T12:08:53.417790+0000","last_fullsized":"2023-12-17T12:08:53.417790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:16:19.492428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800682+0000","last_change":"2023-12-17T12:08:49.188478+0000","last_active":"2023-12-17T12:08:50.800682+0000","last_peered":"2023-12-17T12:08:50.800682+0000","last_clean":"2023-12-17T12:08:50.800682+0000","last_became_active":"2023-12-17T12:08:49.188161+0000","last_became_peered":"2023-12-17T12:08:49.188161+0000","last_unstale":"2023-12-17T12:08:50.800682+0000","last_undegraded":"2023-12-17T12:08:50.800682+0000","last_fullsized":"2023-12-17T12:08:50.800682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:02:14.484656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:55.434379+0000","last_change":"2023-12-17T12:08:48.393092+0000","last_active":"2023-12-17T12:08:55.434379+0000","last_peered":"2023-12-17T12:08:55.434379+0000","last_clean":"2023-12-17T12:08:55.434379+0000","last_became_active":"2023-12-17T12:08:48.392702+0000","last_became_peered":"2023-12-17T12:08:48.392702+0000","last_unstale":"2023-12-17T12:08:55.434379+0000","last_undegraded":"2023-12-17T12:08:55.434379+0000","last_fullsized":"2023-12-17T12:08:55.434379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:53:00.261870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":977,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800064+0000","last_change":"2023-12-17T12:08:49.181367+0000","last_active":"2023-12-17T12:08:50.800064+0000","last_peered":"2023-12-17T12:08:50.800064+0000","last_clean":"2023-12-17T12:08:50.800064+0000","last_became_active":"2023-12-17T12:08:49.180841+0000","last_became_peered":"2023-12-17T12:08:49.180841+0000","last_unstale":"2023-12-17T12:08:50.800064+0000","last_undegraded":"2023-12-17T12:08:50.800064+0000","last_fullsized":"2023-12-17T12:08:50.800064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:57:33.920366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.430590+0000","last_change":"2023-12-17T12:08:48.398863+0000","last_active":"2023-12-17T12:08:54.430590+0000","last_peered":"2023-12-17T12:08:54.430590+0000","last_clean":"2023-12-17T12:08:54.430590+0000","last_became_active":"2023-12-17T12:08:48.396178+0000","last_became_peered":"2023-12-17T12:08:48.396178+0000","last_unstale":"2023-12-17T12:08:54.430590+0000","last_undegraded":"2023-12-17T12:08:54.430590+0000","last_fullsized":"2023-12-17T12:08:54.430590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:31:44.593246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800598+0000","last_change":"2023-12-17T12:08:49.190795+0000","last_active":"2023-12-17T12:08:50.800598+0000","last_peered":"2023-12-17T12:08:50.800598+0000","last_clean":"2023-12-17T12:08:50.800598+0000","last_became_active":"2023-12-17T12:08:49.190468+0000","last_became_peered":"2023-12-17T12:08:49.190468+0000","last_unstale":"2023-12-17T12:08:50.800598+0000","last_undegraded":"2023-12-17T12:08:50.800598+0000","last_fullsized":"2023-12-17T12:08:50.800598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:26:37.302852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.891883+0000","last_change":"2023-12-17T12:08:48.391524+0000","last_active":"2023-12-17T12:08:49.891883+0000","last_peered":"2023-12-17T12:08:49.891883+0000","last_clean":"2023-12-17T12:08:49.891883+0000","last_became_active":"2023-12-17T12:08:48.391195+0000","last_became_peered":"2023-12-17T12:08:48.391195+0000","last_unstale":"2023-12-17T12:08:49.891883+0000","last_undegraded":"2023-12-17T12:08:49.891883+0000","last_fullsized":"2023-12-17T12:08:49.891883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:33:55.399168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800953+0000","last_change":"2023-12-17T12:08:49.188376+0000","last_active":"2023-12-17T12:08:50.800953+0000","last_peered":"2023-12-17T12:08:50.800953+0000","last_clean":"2023-12-17T12:08:50.800953+0000","last_became_active":"2023-12-17T12:08:49.188038+0000","last_became_peered":"2023-12-17T12:08:49.188038+0000","last_unstale":"2023-12-17T12:08:50.800953+0000","last_undegraded":"2023-12-17T12:08:50.800953+0000","last_fullsized":"2023-12-17T12:08:50.800953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:27:35.263053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.164120+0000","last_change":"2023-12-17T12:08:48.401451+0000","last_active":"2023-12-17T12:08:49.164120+0000","last_peered":"2023-12-17T12:08:49.164120+0000","last_clean":"2023-12-17T12:08:49.164120+0000","last_became_active":"2023-12-17T12:08:48.401100+0000","last_became_peered":"2023-12-17T12:08:48.401100+0000","last_unstale":"2023-12-17T12:08:49.164120+0000","last_undegraded":"2023-12-17T12:08:49.164120+0000","last_fullsized":"2023-12-17T12:08:49.164120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:38:48.372997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801722+0000","last_change":"2023-12-17T12:08:49.194033+0000","last_active":"2023-12-17T12:08:50.801722+0000","last_peered":"2023-12-17T12:08:50.801722+0000","last_clean":"2023-12-17T12:08:50.801722+0000","last_became_active":"2023-12-17T12:08:49.192344+0000","last_became_peered":"2023-12-17T12:08:49.192344+0000","last_unstale":"2023-12-17T12:08:50.801722+0000","last_undegraded":"2023-12-17T12:08:50.801722+0000","last_fullsized":"2023-12-17T12:08:50.801722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:00:40.106655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800354+0000","last_change":"2023-12-17T12:08:48.397060+0000","last_active":"2023-12-17T12:08:50.800354+0000","last_peered":"2023-12-17T12:08:50.800354+0000","last_clean":"2023-12-17T12:08:50.800354+0000","last_became_active":"2023-12-17T12:08:48.396704+0000","last_became_peered":"2023-12-17T12:08:48.396704+0000","last_unstale":"2023-12-17T12:08:50.800354+0000","last_undegraded":"2023-12-17T12:08:50.800354+0000","last_fullsized":"2023-12-17T12:08:50.800354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:31:08.882089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801631+0000","last_change":"2023-12-17T12:08:49.183915+0000","last_active":"2023-12-17T12:08:50.801631+0000","last_peered":"2023-12-17T12:08:50.801631+0000","last_clean":"2023-12-17T12:08:50.801631+0000","last_became_active":"2023-12-17T12:08:49.183615+0000","last_became_peered":"2023-12-17T12:08:49.183615+0000","last_unstale":"2023-12-17T12:08:50.801631+0000","last_undegraded":"2023-12-17T12:08:50.801631+0000","last_fullsized":"2023-12-17T12:08:50.801631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:29:09.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800390+0000","last_change":"2023-12-17T12:08:49.190952+0000","last_active":"2023-12-17T12:08:50.800390+0000","last_peered":"2023-12-17T12:08:50.800390+0000","last_clean":"2023-12-17T12:08:50.800390+0000","last_became_active":"2023-12-17T12:08:49.190586+0000","last_became_peered":"2023-12-17T12:08:49.190586+0000","last_unstale":"2023-12-17T12:08:50.800390+0000","last_undegraded":"2023-12-17T12:08:50.800390+0000","last_fullsized":"2023-12-17T12:08:50.800390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:44:00.972429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.430383+0000","last_change":"2023-12-17T12:08:48.409488+0000","last_active":"2023-12-17T12:08:54.430383+0000","last_peered":"2023-12-17T12:08:54.430383+0000","last_clean":"2023-12-17T12:08:54.430383+0000","last_became_active":"2023-12-17T12:08:48.408703+0000","last_became_peered":"2023-12-17T12:08:48.408703+0000","last_unstale":"2023-12-17T12:08:54.430383+0000","last_undegraded":"2023-12-17T12:08:54.430383+0000","last_fullsized":"2023-12-17T12:08:54.430383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:33:14.861530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800861+0000","last_change":"2023-12-17T12:08:49.197960+0000","last_active":"2023-12-17T12:08:50.800861+0000","last_peered":"2023-12-17T12:08:50.800861+0000","last_clean":"2023-12-17T12:08:50.800861+0000","last_became_active":"2023-12-17T12:08:49.196484+0000","last_became_peered":"2023-12-17T12:08:49.196484+0000","last_unstale":"2023-12-17T12:08:50.800861+0000","last_undegraded":"2023-12-17T12:08:50.800861+0000","last_fullsized":"2023-12-17T12:08:50.800861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:53:50.293026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:55.427998+0000","last_change":"2023-12-17T12:08:48.387589+0000","last_active":"2023-12-17T12:08:55.427998+0000","last_peered":"2023-12-17T12:08:55.427998+0000","last_clean":"2023-12-17T12:08:55.427998+0000","last_became_active":"2023-12-17T12:08:48.387258+0000","last_became_peered":"2023-12-17T12:08:48.387258+0000","last_unstale":"2023-12-17T12:08:55.427998+0000","last_undegraded":"2023-12-17T12:08:55.427998+0000","last_fullsized":"2023-12-17T12:08:55.427998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:29:30.815005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":318,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-17T12:08:49.167785+0000","last_change":"2023-12-17T12:07:42.894663+0000","last_active":"2023-12-17T12:08:49.167785+0000","last_peered":"2023-12-17T12:08:49.167785+0000","last_clean":"2023-12-17T12:08:49.167785+0000","last_became_active":"2023-12-17T12:07:42.894199+0000","last_became_peered":"2023-12-17T12:07:42.894199+0000","last_unstale":"2023-12-17T12:08:49.167785+0000","last_undegraded":"2023-12-17T12:08:49.167785+0000","last_fullsized":"2023-12-17T12:08:49.167785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:04:41.531220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:04:41.531220+0000","last_clean_scrub_stamp":"2023-12-17T12:04:41.531220+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:07:49.358197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.165667+0000","last_change":"2023-12-17T12:08:49.195358+0000","last_active":"2023-12-17T12:08:50.165667+0000","last_peered":"2023-12-17T12:08:50.165667+0000","last_clean":"2023-12-17T12:08:50.165667+0000","last_became_active":"2023-12-17T12:08:49.194996+0000","last_became_peered":"2023-12-17T12:08:49.194996+0000","last_unstale":"2023-12-17T12:08:50.165667+0000","last_undegraded":"2023-12-17T12:08:50.165667+0000","last_fullsized":"2023-12-17T12:08:50.165667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:23:10.016755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.429330+0000","last_change":"2023-12-17T12:08:48.394937+0000","last_active":"2023-12-17T12:08:54.429330+0000","last_peered":"2023-12-17T12:08:54.429330+0000","last_clean":"2023-12-17T12:08:54.429330+0000","last_became_active":"2023-12-17T12:08:48.394590+0000","last_became_peered":"2023-12-17T12:08:48.394590+0000","last_unstale":"2023-12-17T12:08:54.429330+0000","last_undegraded":"2023-12-17T12:08:54.429330+0000","last_fullsized":"2023-12-17T12:08:54.429330+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:24:49.088883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.167099+0000","last_change":"2023-12-17T12:08:48.396769+0000","last_active":"2023-12-17T12:08:50.167099+0000","last_peered":"2023-12-17T12:08:50.167099+0000","last_clean":"2023-12-17T12:08:50.167099+0000","last_became_active":"2023-12-17T12:08:48.395984+0000","last_became_peered":"2023-12-17T12:08:48.395984+0000","last_unstale":"2023-12-17T12:08:50.167099+0000","last_undegraded":"2023-12-17T12:08:50.167099+0000","last_fullsized":"2023-12-17T12:08:50.167099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:34:59.872771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.165878+0000","last_change":"2023-12-17T12:08:49.194598+0000","last_active":"2023-12-17T12:08:50.165878+0000","last_peered":"2023-12-17T12:08:50.165878+0000","last_clean":"2023-12-17T12:08:50.165878+0000","last_became_active":"2023-12-17T12:08:49.189450+0000","last_became_peered":"2023-12-17T12:08:49.189450+0000","last_unstale":"2023-12-17T12:08:50.165878+0000","last_undegraded":"2023-12-17T12:08:50.165878+0000","last_fullsized":"2023-12-17T12:08:50.165878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:45:57.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.428399+0000","last_change":"2023-12-17T12:08:48.407902+0000","last_active":"2023-12-17T12:08:54.428399+0000","last_peered":"2023-12-17T12:08:54.428399+0000","last_clean":"2023-12-17T12:08:54.428399+0000","last_became_active":"2023-12-17T12:08:48.407601+0000","last_became_peered":"2023-12-17T12:08:48.407601+0000","last_unstale":"2023-12-17T12:08:54.428399+0000","last_undegraded":"2023-12-17T12:08:54.428399+0000","last_fullsized":"2023-12-17T12:08:54.428399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:56:34.626900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800934+0000","last_change":"2023-12-17T12:08:49.196489+0000","last_active":"2023-12-17T12:08:50.800934+0000","last_peered":"2023-12-17T12:08:50.800934+0000","last_clean":"2023-12-17T12:08:50.800934+0000","last_became_active":"2023-12-17T12:08:49.196063+0000","last_became_peered":"2023-12-17T12:08:49.196063+0000","last_unstale":"2023-12-17T12:08:50.800934+0000","last_undegraded":"2023-12-17T12:08:50.800934+0000","last_fullsized":"2023-12-17T12:08:50.800934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:16:08.658987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.421014+0000","last_change":"2023-12-17T12:08:48.400075+0000","last_active":"2023-12-17T12:08:53.421014+0000","last_peered":"2023-12-17T12:08:53.421014+0000","last_clean":"2023-12-17T12:08:53.421014+0000","last_became_active":"2023-12-17T12:08:48.398728+0000","last_became_peered":"2023-12-17T12:08:48.398728+0000","last_unstale":"2023-12-17T12:08:53.421014+0000","last_undegraded":"2023-12-17T12:08:53.421014+0000","last_fullsized":"2023-12-17T12:08:53.421014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:10:21.065038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799938+0000","last_change":"2023-12-17T12:08:49.193200+0000","last_active":"2023-12-17T12:08:50.799938+0000","last_peered":"2023-12-17T12:08:50.799938+0000","last_clean":"2023-12-17T12:08:50.799938+0000","last_became_active":"2023-12-17T12:08:49.192855+0000","last_became_peered":"2023-12-17T12:08:49.192855+0000","last_unstale":"2023-12-17T12:08:50.799938+0000","last_undegraded":"2023-12-17T12:08:50.799938+0000","last_fullsized":"2023-12-17T12:08:50.799938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:14:09.089956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.418408+0000","last_change":"2023-12-17T12:08:48.404661+0000","last_active":"2023-12-17T12:08:53.418408+0000","last_peered":"2023-12-17T12:08:53.418408+0000","last_clean":"2023-12-17T12:08:53.418408+0000","last_became_active":"2023-12-17T12:08:48.404336+0000","last_became_peered":"2023-12-17T12:08:48.404336+0000","last_unstale":"2023-12-17T12:08:53.418408+0000","last_undegraded":"2023-12-17T12:08:53.418408+0000","last_fullsized":"2023-12-17T12:08:53.418408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:37:20.863966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166701+0000","last_change":"2023-12-17T12:08:49.181169+0000","last_active":"2023-12-17T12:08:50.166701+0000","last_peered":"2023-12-17T12:08:50.166701+0000","last_clean":"2023-12-17T12:08:50.166701+0000","last_became_active":"2023-12-17T12:08:49.177802+0000","last_became_peered":"2023-12-17T12:08:49.177802+0000","last_unstale":"2023-12-17T12:08:50.166701+0000","last_undegraded":"2023-12-17T12:08:50.166701+0000","last_fullsized":"2023-12-17T12:08:50.166701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:56:50.122246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.417528+0000","last_change":"2023-12-17T12:08:48.407865+0000","last_active":"2023-12-17T12:08:53.417528+0000","last_peered":"2023-12-17T12:08:53.417528+0000","last_clean":"2023-12-17T12:08:53.417528+0000","last_became_active":"2023-12-17T12:08:48.407580+0000","last_became_peered":"2023-12-17T12:08:48.407580+0000","last_unstale":"2023-12-17T12:08:53.417528+0000","last_undegraded":"2023-12-17T12:08:53.417528+0000","last_fullsized":"2023-12-17T12:08:53.417528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:10:20.505630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800530+0000","last_change":"2023-12-17T12:08:49.192724+0000","last_active":"2023-12-17T12:08:50.800530+0000","last_peered":"2023-12-17T12:08:50.800530+0000","last_clean":"2023-12-17T12:08:50.800530+0000","last_became_active":"2023-12-17T12:08:49.188658+0000","last_became_peered":"2023-12-17T12:08:49.188658+0000","last_unstale":"2023-12-17T12:08:50.800530+0000","last_undegraded":"2023-12-17T12:08:50.800530+0000","last_fullsized":"2023-12-17T12:08:50.800530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:00:17.493003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:51.314265+0000","last_change":"2023-12-17T12:08:48.404717+0000","last_active":"2023-12-17T12:08:51.314265+0000","last_peered":"2023-12-17T12:08:51.314265+0000","last_clean":"2023-12-17T12:08:51.314265+0000","last_became_active":"2023-12-17T12:08:48.404426+0000","last_became_peered":"2023-12-17T12:08:48.404426+0000","last_unstale":"2023-12-17T12:08:51.314265+0000","last_undegraded":"2023-12-17T12:08:51.314265+0000","last_fullsized":"2023-12-17T12:08:51.314265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:35:28.090855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800627+0000","last_change":"2023-12-17T12:08:49.191911+0000","last_active":"2023-12-17T12:08:50.800627+0000","last_peered":"2023-12-17T12:08:50.800627+0000","last_clean":"2023-12-17T12:08:50.800627+0000","last_became_active":"2023-12-17T12:08:49.191573+0000","last_became_peered":"2023-12-17T12:08:49.191573+0000","last_unstale":"2023-12-17T12:08:50.800627+0000","last_undegraded":"2023-12-17T12:08:50.800627+0000","last_fullsized":"2023-12-17T12:08:50.800627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:39:35.354221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:53.421954+0000","last_change":"2023-12-17T12:08:48.401253+0000","last_active":"2023-12-17T12:08:53.421954+0000","last_peered":"2023-12-17T12:08:53.421954+0000","last_clean":"2023-12-17T12:08:53.421954+0000","last_became_active":"2023-12-17T12:08:48.399675+0000","last_became_peered":"2023-12-17T12:08:48.399675+0000","last_unstale":"2023-12-17T12:08:53.421954+0000","last_undegraded":"2023-12-17T12:08:53.421954+0000","last_fullsized":"2023-12-17T12:08:53.421954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:15:47.969526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800959+0000","last_change":"2023-12-17T12:08:49.191906+0000","last_active":"2023-12-17T12:08:50.800959+0000","last_peered":"2023-12-17T12:08:50.800959+0000","last_clean":"2023-12-17T12:08:50.800959+0000","last_became_active":"2023-12-17T12:08:49.191572+0000","last_became_peered":"2023-12-17T12:08:49.191572+0000","last_unstale":"2023-12-17T12:08:50.800959+0000","last_undegraded":"2023-12-17T12:08:50.800959+0000","last_fullsized":"2023-12-17T12:08:50.800959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:56:05.670034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:54.430335+0000","last_change":"2023-12-17T12:08:48.399004+0000","last_active":"2023-12-17T12:08:54.430335+0000","last_peered":"2023-12-17T12:08:54.430335+0000","last_clean":"2023-12-17T12:08:54.430335+0000","last_became_active":"2023-12-17T12:08:48.398453+0000","last_became_peered":"2023-12-17T12:08:48.398453+0000","last_unstale":"2023-12-17T12:08:54.430335+0000","last_undegraded":"2023-12-17T12:08:54.430335+0000","last_fullsized":"2023-12-17T12:08:54.430335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:30:53.767008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800987+0000","last_change":"2023-12-17T12:08:49.178859+0000","last_active":"2023-12-17T12:08:50.800987+0000","last_peered":"2023-12-17T12:08:50.800987+0000","last_clean":"2023-12-17T12:08:50.800987+0000","last_became_active":"2023-12-17T12:08:49.178119+0000","last_became_peered":"2023-12-17T12:08:49.178119+0000","last_unstale":"2023-12-17T12:08:50.800987+0000","last_undegraded":"2023-12-17T12:08:50.800987+0000","last_fullsized":"2023-12-17T12:08:50.800987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:32:40.758242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801506+0000","last_change":"2023-12-17T12:08:48.407249+0000","last_active":"2023-12-17T12:08:50.801506+0000","last_peered":"2023-12-17T12:08:50.801506+0000","last_clean":"2023-12-17T12:08:50.801506+0000","last_became_active":"2023-12-17T12:08:48.406572+0000","last_became_peered":"2023-12-17T12:08:48.406572+0000","last_unstale":"2023-12-17T12:08:50.801506+0000","last_undegraded":"2023-12-17T12:08:50.801506+0000","last_fullsized":"2023-12-17T12:08:50.801506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:19:07.751233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:52.426195+0000","last_change":"2023-12-17T12:08:49.192590+0000","last_active":"2023-12-17T12:08:52.426195+0000","last_peered":"2023-12-17T12:08:52.426195+0000","last_clean":"2023-12-17T12:08:52.426195+0000","last_became_active":"2023-12-17T12:08:49.192250+0000","last_became_peered":"2023-12-17T12:08:49.192250+0000","last_unstale":"2023-12-17T12:08:52.426195+0000","last_undegraded":"2023-12-17T12:08:52.426195+0000","last_fullsized":"2023-12-17T12:08:52.426195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:51:20.599412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166058+0000","last_change":"2023-12-17T12:08:48.406799+0000","last_active":"2023-12-17T12:08:50.166058+0000","last_peered":"2023-12-17T12:08:50.166058+0000","last_clean":"2023-12-17T12:08:50.166058+0000","last_became_active":"2023-12-17T12:08:48.406423+0000","last_became_peered":"2023-12-17T12:08:48.406423+0000","last_unstale":"2023-12-17T12:08:50.166058+0000","last_undegraded":"2023-12-17T12:08:50.166058+0000","last_fullsized":"2023-12-17T12:08:50.166058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:43:55.600399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801260+0000","last_change":"2023-12-17T12:08:49.182840+0000","last_active":"2023-12-17T12:08:50.801260+0000","last_peered":"2023-12-17T12:08:50.801260+0000","last_clean":"2023-12-17T12:08:50.801260+0000","last_became_active":"2023-12-17T12:08:49.181562+0000","last_became_peered":"2023-12-17T12:08:49.181562+0000","last_unstale":"2023-12-17T12:08:50.801260+0000","last_undegraded":"2023-12-17T12:08:50.801260+0000","last_fullsized":"2023-12-17T12:08:50.801260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:01:41.059044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800478+0000","last_change":"2023-12-17T12:08:49.193449+0000","last_active":"2023-12-17T12:08:50.800478+0000","last_peered":"2023-12-17T12:08:50.800478+0000","last_clean":"2023-12-17T12:08:50.800478+0000","last_became_active":"2023-12-17T12:08:49.192942+0000","last_became_peered":"2023-12-17T12:08:49.192942+0000","last_unstale":"2023-12-17T12:08:50.800478+0000","last_undegraded":"2023-12-17T12:08:50.800478+0000","last_fullsized":"2023-12-17T12:08:50.800478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:42:51.021808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799882+0000","last_change":"2023-12-17T12:08:49.193515+0000","last_active":"2023-12-17T12:08:50.799882+0000","last_peered":"2023-12-17T12:08:50.799882+0000","last_clean":"2023-12-17T12:08:50.799882+0000","last_became_active":"2023-12-17T12:08:49.193232+0000","last_became_peered":"2023-12-17T12:08:49.193232+0000","last_unstale":"2023-12-17T12:08:50.799882+0000","last_undegraded":"2023-12-17T12:08:50.799882+0000","last_fullsized":"2023-12-17T12:08:50.799882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:26:59.948350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799983+0000","last_change":"2023-12-17T12:08:49.197110+0000","last_active":"2023-12-17T12:08:50.799983+0000","last_peered":"2023-12-17T12:08:50.799983+0000","last_clean":"2023-12-17T12:08:50.799983+0000","last_became_active":"2023-12-17T12:08:49.196712+0000","last_became_peered":"2023-12-17T12:08:49.196712+0000","last_unstale":"2023-12-17T12:08:50.799983+0000","last_undegraded":"2023-12-17T12:08:50.799983+0000","last_fullsized":"2023-12-17T12:08:50.799983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:21:40.555247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.799414+0000","last_change":"2023-12-17T12:08:49.192021+0000","last_active":"2023-12-17T12:08:50.799414+0000","last_peered":"2023-12-17T12:08:50.799414+0000","last_clean":"2023-12-17T12:08:50.799414+0000","last_became_active":"2023-12-17T12:08:49.191696+0000","last_became_peered":"2023-12-17T12:08:49.191696+0000","last_unstale":"2023-12-17T12:08:50.799414+0000","last_undegraded":"2023-12-17T12:08:50.799414+0000","last_fullsized":"2023-12-17T12:08:50.799414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:27:40.425617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166136+0000","last_change":"2023-12-17T12:08:49.196208+0000","last_active":"2023-12-17T12:08:50.166136+0000","last_peered":"2023-12-17T12:08:50.166136+0000","last_clean":"2023-12-17T12:08:50.166136+0000","last_became_active":"2023-12-17T12:08:49.195900+0000","last_became_peered":"2023-12-17T12:08:49.195900+0000","last_unstale":"2023-12-17T12:08:50.166136+0000","last_undegraded":"2023-12-17T12:08:50.166136+0000","last_fullsized":"2023-12-17T12:08:50.166136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:07:05.201192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800886+0000","last_change":"2023-12-17T12:08:49.191147+0000","last_active":"2023-12-17T12:08:50.800886+0000","last_peered":"2023-12-17T12:08:50.800886+0000","last_clean":"2023-12-17T12:08:50.800886+0000","last_became_active":"2023-12-17T12:08:49.190804+0000","last_became_peered":"2023-12-17T12:08:49.190804+0000","last_unstale":"2023-12-17T12:08:50.800886+0000","last_undegraded":"2023-12-17T12:08:50.800886+0000","last_fullsized":"2023-12-17T12:08:50.800886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:09:40.551342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.801114+0000","last_change":"2023-12-17T12:08:49.184707+0000","last_active":"2023-12-17T12:08:50.801114+0000","last_peered":"2023-12-17T12:08:50.801114+0000","last_clean":"2023-12-17T12:08:50.801114+0000","last_became_active":"2023-12-17T12:08:49.183644+0000","last_became_peered":"2023-12-17T12:08:49.183644+0000","last_unstale":"2023-12-17T12:08:50.801114+0000","last_undegraded":"2023-12-17T12:08:50.801114+0000","last_fullsized":"2023-12-17T12:08:50.801114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:30:43.845054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800791+0000","last_change":"2023-12-17T12:08:49.195299+0000","last_active":"2023-12-17T12:08:50.800791+0000","last_peered":"2023-12-17T12:08:50.800791+0000","last_clean":"2023-12-17T12:08:50.800791+0000","last_became_active":"2023-12-17T12:08:49.194315+0000","last_became_peered":"2023-12-17T12:08:49.194315+0000","last_unstale":"2023-12-17T12:08:50.800791+0000","last_undegraded":"2023-12-17T12:08:50.800791+0000","last_fullsized":"2023-12-17T12:08:50.800791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:33:26.056073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800236+0000","last_change":"2023-12-17T12:08:49.196145+0000","last_active":"2023-12-17T12:08:50.800236+0000","last_peered":"2023-12-17T12:08:50.800236+0000","last_clean":"2023-12-17T12:08:50.800236+0000","last_became_active":"2023-12-17T12:08:49.195893+0000","last_became_peered":"2023-12-17T12:08:49.195893+0000","last_unstale":"2023-12-17T12:08:50.800236+0000","last_undegraded":"2023-12-17T12:08:50.800236+0000","last_fullsized":"2023-12-17T12:08:50.800236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:33:23.868301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800225+0000","last_change":"2023-12-17T12:08:49.193113+0000","last_active":"2023-12-17T12:08:50.800225+0000","last_peered":"2023-12-17T12:08:50.800225+0000","last_clean":"2023-12-17T12:08:50.800225+0000","last_became_active":"2023-12-17T12:08:49.191730+0000","last_became_peered":"2023-12-17T12:08:49.191730+0000","last_unstale":"2023-12-17T12:08:50.800225+0000","last_undegraded":"2023-12-17T12:08:50.800225+0000","last_fullsized":"2023-12-17T12:08:50.800225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:45:33.858040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800050+0000","last_change":"2023-12-17T12:08:49.193401+0000","last_active":"2023-12-17T12:08:50.800050+0000","last_peered":"2023-12-17T12:08:50.800050+0000","last_clean":"2023-12-17T12:08:50.800050+0000","last_became_active":"2023-12-17T12:08:49.192993+0000","last_became_peered":"2023-12-17T12:08:49.192993+0000","last_unstale":"2023-12-17T12:08:50.800050+0000","last_undegraded":"2023-12-17T12:08:50.800050+0000","last_fullsized":"2023-12-17T12:08:50.800050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:21:16.594494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.166016+0000","last_change":"2023-12-17T12:08:49.200282+0000","last_active":"2023-12-17T12:08:50.166016+0000","last_peered":"2023-12-17T12:08:50.166016+0000","last_clean":"2023-12-17T12:08:50.166016+0000","last_became_active":"2023-12-17T12:08:49.199788+0000","last_became_peered":"2023-12-17T12:08:49.199788+0000","last_unstale":"2023-12-17T12:08:50.166016+0000","last_undegraded":"2023-12-17T12:08:50.166016+0000","last_fullsized":"2023-12-17T12:08:50.166016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:07:19.164767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-17T12:08:50.800199+0000","last_change":"2023-12-17T12:08:49.185712+0000","last_active":"2023-12-17T12:08:50.800199+0000","last_peered":"2023-12-17T12:08:50.800199+0000","last_clean":"2023-12-17T12:08:50.800199+0000","last_became_active":"2023-12-17T12:08:49.185417+0000","last_became_peered":"2023-12-17T12:08:49.185417+0000","last_unstale":"2023-12-17T12:08:50.800199+0000","last_undegraded":"2023-12-17T12:08:50.800199+0000","last_fullsized":"2023-12-17T12:08:50.800199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:28:43.076574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246978,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5611,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":122,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":33096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481805,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1017058,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54800000000000004}]},{"osd":2,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.72499999999999998}]},{"osd":3,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77000000000000002}]},{"osd":4,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.58099999999999996}]},{"osd":5,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.89000000000000001}]},{"osd":7,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.82599999999999996}]},{"osd":8,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.81999999999999995}]},{"osd":9,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.86899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]}]},{"osd":9,"up_from":64,"seq":274877906965,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":424940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.442},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.85099999999999998}]},{"osd":2,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":13.166,"5min":13.166,"15min":13.166},"last":0.58799999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.66500000000000004}]},{"osd":4,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.75}]},{"osd":5,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":7.1920000000000002,"5min":7.1920000000000002,"15min":7.1920000000000002},"last":0.621},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.64200000000000002}]},{"osd":6,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.88800000000000001}]},{"osd":8,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":13.18,"5min":13.18,"15min":13.18},"last":0.78700000000000003}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.87},{"interface":"front","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":13.212,"5min":13.212,"15min":13.212},"last":0.93400000000000005}]}]},{"osd":8,"up_from":56,"seq":240518168602,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":424748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":7.0350000000000001,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":7.0119999999999996,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":1.135}]},{"osd":2,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.90300000000000002}]},{"osd":3,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":7.0599999999999996,"5min":7.0599999999999996,"15min":7.0599999999999996},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":6.992,"5min":6.992,"15min":6.992},"last":1.097}]},{"osd":4,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":1.179},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.2549999999999999}]},{"osd":5,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.232}]},{"osd":6,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.47},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.276}]},{"osd":7,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.331},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.4470000000000001}]},{"osd":9,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.4910000000000001}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.5149999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.631}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397535,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1016463,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92200000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57599999999999996}]},{"osd":1,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.496,"15min":0.48099999999999998},"min":{"1min":0.38900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.94699999999999995}]},{"osd":2,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.78900000000000003,"15min":0.745},"min":{"1min":0.52800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":7.1070000000000002,"5min":7.1070000000000002,"15min":7.1070000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.96899999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95899999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78300000000000003}]},{"osd":4,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94999999999999996,"5min":1.119,"15min":1.119},"last":0.73999999999999999}]},{"osd":5,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.84899999999999998,"15min":0.80100000000000005},"min":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":7.1669999999999998,"5min":7.1669999999999998,"15min":7.1669999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.069}]},{"osd":6,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.02},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.84199999999999997,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0780000000000001,"5min":1.21,"15min":1.21},"last":0.995}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.69799999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.143,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.125},"min":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.101,"5min":7.1369999999999996,"15min":7.1369999999999996},"last":1.0880000000000001}]},{"osd":9,"last update":"Sun Dec 17 12:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.048},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.167}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.1930000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]}]},{"osd":6,"up_from":44,"seq":188978561060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":424212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.67100000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68000000000000005}]},{"osd":1,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.72799999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.77600000000000002,"15min":0.746},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":5.6740000000000004,"5min":5.6740000000000004,"15min":5.6740000000000004},"last":0.59699999999999998}]},{"osd":3,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.45500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.80200000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0529999999999999,"5min":1.167,"15min":1.167},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80700000000000005,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.85799999999999998,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.72899999999999998}]},{"osd":7,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.91500000000000004}]},{"osd":8,"last update":"Sun Dec 17 12:07:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.755}]},{"osd":9,"last update":"Sun Dec 17 12:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.93700000000000006}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.61699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":425734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.58599999999999997,"15min":0.56100000000000005},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.62,"15min":0.58599999999999997},"min":{"1min":0.53800000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.626}]},{"osd":2,"last update":"Sun Dec 17 12:08:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63800000000000001,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0549999999999999,"5min":1.111,"15min":1.111},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63400000000000001,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.097,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1}]},{"osd":3,"last update":"Sun Dec 17 12:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.623,"15min":0.59199999999999997},"min":{"1min":0.53900000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.622,"15min":0.58999999999999997},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Sun Dec 17 12:08:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79200000000000004}]},{"osd":5,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.69699999999999995,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.70199999999999996}]},{"osd":6,"last update":"Sun Dec 17 12:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.59299999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.751,"15min":0.73199999999999998},"min":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.84099999999999997}]},{"osd":7,"last update":"Sun Dec 17 12:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68899999999999995,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81599999999999995}]},{"osd":8,"last update":"Sun Dec 17 12:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":1.054},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0649999999999999}]},{"osd":9,"last update":"Sun Dec 17 12:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.018}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.92300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":425600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Dec 17 12:08:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.63100000000000001,"15min":0.61099999999999999},"min":{"1min":0.51400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.157},"last":0.504},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.66200000000000003,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Sun Dec 17 12:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65900000000000003,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65000000000000002,"15min":0.59999999999999998},"min":{"1min":0.55900000000000005,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Dec 17 12:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.375,"5min":0.315,"15min":0.315},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79600000000000004}]},{"osd":4,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.64500000000000002},"min":{"1min":0.504,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.077,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.58299999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.171,"5min":1.248,"15min":1.248},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Dec 17 12:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0329999999999999,"5min":1.083,"15min":1.083},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.51800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.83399999999999996},"min":{"1min":0.71999999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.871,"15min":0.86299999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.81599999999999995}]},{"osd":8,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.91100000000000003}]},{"osd":9,"last update":"Sun Dec 17 12:08:20 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.82799999999999996}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.98499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.52400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.48199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.58699999999999997}]},{"osd":1,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.66800000000000004,"15min":0.621},"min":{"1min":0.54500000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98099999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Dec 17 12:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.627}]},{"osd":4,"last update":"Sun Dec 17 12:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.69199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95799999999999996,"5min":1.181,"15min":1.181},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94699999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.82399999999999995}]},{"osd":6,"last update":"Sun Dec 17 12:08:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.73199999999999998,"15min":0.70699999999999996},"min":{"1min":0.67800000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92300000000000004,"5min":1.214,"15min":1.214},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.75900000000000001}]},{"osd":8,"last update":"Sun Dec 17 12:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88600000000000001}]},{"osd":9,"last update":"Sun Dec 17 12:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.80500000000000005}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.96299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":424316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41599999999999998,"15min":0.39200000000000002},"min":{"1min":0.34399999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.44700000000000001,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.51600000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.63800000000000001,"15min":0.60799999999999998},"min":{"1min":0.46200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.54600000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80300000000000005}]},{"osd":2,"last update":"Sun Dec 17 12:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.69499999999999995,"15min":0.67600000000000005},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.69199999999999995,"15min":0.66100000000000003},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.86099999999999999}]},{"osd":4,"last update":"Sun Dec 17 12:08:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.67500000000000004,"15min":0.63500000000000001},"min":{"1min":0.623,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.66400000000000003}]},{"osd":5,"last update":"Sun Dec 17 12:07:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.876},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75,"15min":0.72799999999999998},"min":{"1min":0.52700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Dec 17 12:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.46400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.754},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.51500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84999999999999998,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90300000000000002}]},{"osd":8,"last update":"Sun Dec 17 12:07:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.71999999999999997}]},{"osd":9,"last update":"Sun Dec 17 12:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.95099999999999996}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.006}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953516,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":424766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.49299999999999999,"15min":0.46700000000000003},"min":{"1min":0.36399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.45900000000000002,"15min":0.435},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.77600000000000002,"15min":0.82599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.087,"5min":5.125,"15min":5.125},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.64300000000000002,"15min":0.61599999999999999},"min":{"1min":0.46000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65700000000000003,"15min":0.629},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Dec 17 12:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68400000000000005,"15min":0.65900000000000003},"min":{"1min":0.59599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.874},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68799999999999994,"15min":0.66800000000000004},"min":{"1min":0.57599999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90600000000000003,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.70999999999999996}]},{"osd":6,"last update":"Sun Dec 17 12:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.93899999999999995,"15min":0.96799999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.389,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.754,"15min":0.746},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.63200000000000001}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.70299999999999996,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.81799999999999995}]},{"osd":8,"last update":"Sun Dec 17 12:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.74199999999999999}]},{"osd":9,"last update":"Sun Dec 17 12:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.91200000000000003}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.97999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":5,"up_from":37,"seq":158913789993,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":1016089,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94999999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.64200000000000002}]},{"osd":1,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.55100000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89400000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.752},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66200000000000003,"15min":0.63500000000000001},"min":{"1min":0.53400000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.876,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.39700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95099999999999996,"5min":1.097,"15min":1.097},"last":0.623}]},{"osd":4,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":0.92700000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.52600000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95499999999999996,"5min":1.048,"15min":1.048},"last":0.81200000000000006}]},{"osd":6,"last update":"Sun Dec 17 12:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.62,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.94799999999999995}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":1.0589999999999999,"5min":1.117,"15min":1.117},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":0.83899999999999997},"min":{"1min":0.63500000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.95899999999999996,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.105,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.875},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.83999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1419999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.79200000000000004}]},{"osd":9,"last update":"Sun Dec 17 12:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.96699999999999997}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":1.01},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.91000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34109,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34109,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-17T12:08:59.456 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-17T12:08:59.457 INFO:tasks.cephadm:Setting up client nodes... 2023-12-17T12:08:59.457 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-17T12:08:59.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:08:59 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2385363428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:59.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:08:59 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2385363428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:59.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:08:59 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2385363428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:08:59.918 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:00.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:00 smithi141 ceph-mon[41225]: pgmap v270: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-12-17T12:09:00.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:00 smithi141 ceph-mon[41225]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:00.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:00 smithi141 ceph-mon[41225]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:00.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:00 smithi192 ceph-mon[41899]: pgmap v270: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-12-17T12:09:00.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:00 smithi192 ceph-mon[41899]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:00.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:00 smithi192 ceph-mon[41899]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:00.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:00 smithi027 ceph-mon[36871]: pgmap v270: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-12-17T12:09:00.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:00 smithi027 ceph-mon[36871]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:00.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:00 smithi027 ceph-mon[36871]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 -- 172.21.15.27:0/1030034016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d0103180 msgr2=0x7f63d0109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 --2- 172.21.15.27:0/1030034016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d0103180 0x7f63d0109a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f63c800b0d0 tx=0x7f63c800b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 -- 172.21.15.27:0/1030034016 shutdown_connections 2023-12-17T12:09:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 --2- 172.21.15.27:0/1030034016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d0103180 0x7f63d0109a30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 --2- 172.21.15.27:0/1030034016 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63d01015e0 0x7f63d0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 --2- 172.21.15.27:0/1030034016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63d01027e0 0x7f63d0102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 -- 172.21.15.27:0/1030034016 >> 172.21.15.27:0/1030034016 conn(0x7f63d00fcb80 msgr2=0x7f63d00fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 -- 172.21.15.27:0/1030034016 shutdown_connections 2023-12-17T12:09:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.239+0000 7f63d7e54700 1 -- 172.21.15.27:0/1030034016 wait complete. 2023-12-17T12:09:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.240+0000 7f63d7e54700 1 Processor -- start 2023-12-17T12:09:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d7e54700 1 -- start start 2023-12-17T12:09:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d7e54700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d01015e0 0x7f63d0075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d7e54700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63d01027e0 0x7f63d00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d7e54700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63d0103180 0x7f63d0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d7e54700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00746a0 con 0x7f63d01015e0 2023-12-17T12:09:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d7e54700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0074810 con 0x7f63d01027e0 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d7e54700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d01a15c0 con 0x7f63d0103180 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d5bf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d01015e0 0x7f63d0075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d5bf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d01015e0 0x7f63d0075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59380/0 (socket says 172.21.15.27:59380) 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d5bf0700 1 -- 172.21.15.27:0/3669308294 learned_addr learned my addr 172.21.15.27:0/3669308294 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d63f1700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63d0103180 0x7f63d0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.241+0000 7f63d53ef700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63d01027e0 0x7f63d00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63d5bf0700 1 -- 172.21.15.27:0/3669308294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63d0103180 msgr2=0x7f63d0073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63d5bf0700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63d0103180 0x7f63d0073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63d5bf0700 1 -- 172.21.15.27:0/3669308294 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63d01027e0 msgr2=0x7f63d00739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63d5bf0700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63d01027e0 0x7f63d00739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63d5bf0700 1 -- 172.21.15.27:0/3669308294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c8009d00 con 0x7f63d01015e0 2023-12-17T12:09:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63d5bf0700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d01015e0 0x7f63d0075320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e970 tx=0x7f63cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:01.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63c6ffd700 1 -- 172.21.15.27:0/3669308294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc009790 con 0x7f63d01015e0 2023-12-17T12:09:01.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63c6ffd700 1 -- 172.21.15.27:0/3669308294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63cc00c7e0 con 0x7f63d01015e0 2023-12-17T12:09:01.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63c6ffd700 1 -- 172.21.15.27:0/3669308294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc003bd0 con 0x7f63d01015e0 2023-12-17T12:09:01.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.242+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d01a1850 con 0x7f63d01015e0 2023-12-17T12:09:01.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.243+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d01a1d00 con 0x7f63d01015e0 2023-12-17T12:09:01.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.244+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d0108ac0 con 0x7f63d01015e0 2023-12-17T12:09:01.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.246+0000 7f63c6ffd700 1 -- 172.21.15.27:0/3669308294 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f63cc00c950 con 0x7f63d01015e0 2023-12-17T12:09:01.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.247+0000 7f63c6ffd700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63bc0a1510 0x7f63bc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:01.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.247+0000 7f63c6ffd700 1 -- 172.21.15.27:0/3669308294 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f63cc014070 con 0x7f63d01015e0 2023-12-17T12:09:01.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.248+0000 7f63d53ef700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63bc0a1510 0x7f63bc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:01.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.248+0000 7f63d53ef700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63bc0a1510 0x7f63bc0a39d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f63c0004630 tx=0x7f63c000b3c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:01.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.251+0000 7f63c6ffd700 1 -- 172.21.15.27:0/3669308294 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f63cc096530 con 0x7f63d01015e0 2023-12-17T12:09:01.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.550+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f63d004f290 con 0x7f63d01015e0 2023-12-17T12:09:01.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.553+0000 7f63c6ffd700 1 -- 172.21.15.27:0/3669308294 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f63d004f290 con 0x7f63d01015e0 2023-12-17T12:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2023-12-17T12:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDd5H5l3VLWIBAAv++L8Gqpb7DhSuxJ7EXlWA== 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63bc0a1510 msgr2=0x7f63bc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63bc0a1510 0x7f63bc0a39d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f63c0004630 tx=0x7f63c000b3c0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d01015e0 msgr2=0x7f63d0075320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d01015e0 0x7f63d0075320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e970 tx=0x7f63cc00ec80 comp rx=0 tx=0).stop 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 shutdown_connections 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f63bc0a1510 0x7f63bc0a39d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63d0103180 0x7f63d0073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63d01015e0 0x7f63d0075320 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 --2- 172.21.15.27:0/3669308294 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f63d01027e0 0x7f63d00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 >> 172.21.15.27:0/3669308294 conn(0x7f63d00fcb80 msgr2=0x7f63d0105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.556+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 shutdown_connections 2023-12-17T12:09:01.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:01.557+0000 7f63d7e54700 1 -- 172.21.15.27:0/3669308294 wait complete. 2023-12-17T12:09:01.953 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-17T12:09:01.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-17T12:09:01.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-17T12:09:02.057 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-17T12:09:02.057 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-17T12:09:02.057 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph mgr dump --format=json 2023-12-17T12:09:02.528 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:02.692 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:02 smithi027 ceph-mon[36871]: pgmap v271: 97 pgs: 97 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.8 KiB/s wr, 17 op/s 2023-12-17T12:09:02.692 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:02 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3669308294' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-17T12:09:02.692 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:02 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3669308294' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-17T12:09:02.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:02 smithi141 ceph-mon[41225]: pgmap v271: 97 pgs: 97 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.8 KiB/s wr, 17 op/s 2023-12-17T12:09:02.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:02 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3669308294' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-17T12:09:02.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:02 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3669308294' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-17T12:09:02.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:02 smithi192 ceph-mon[41899]: pgmap v271: 97 pgs: 97 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.8 KiB/s wr, 17 op/s 2023-12-17T12:09:02.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:02 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3669308294' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-17T12:09:02.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:02 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3669308294' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-17T12:09:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 -- 172.21.15.27:0/380257968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3530102040 msgr2=0x7f353010e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 --2- 172.21.15.27:0/380257968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3530102040 0x7f353010e430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f352800b0a0 tx=0x7f352800b3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 -- 172.21.15.27:0/380257968 shutdown_connections 2023-12-17T12:09:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 --2- 172.21.15.27:0/380257968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35301076f0 0x7f3530107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 --2- 172.21.15.27:0/380257968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3530102040 0x7f353010e430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 --2- 172.21.15.27:0/380257968 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3530101680 0x7f3530101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 -- 172.21.15.27:0/380257968 >> 172.21.15.27:0/380257968 conn(0x7f35300787c0 msgr2=0x7f3530078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 -- 172.21.15.27:0/380257968 shutdown_connections 2023-12-17T12:09:03.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.731+0000 7f353743a700 1 -- 172.21.15.27:0/380257968 wait complete. 2023-12-17T12:09:03.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.732+0000 7f353743a700 1 Processor -- start 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.732+0000 7f353743a700 1 -- start start 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.732+0000 7f353743a700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3530101680 0x7f35300738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.732+0000 7f353743a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3530102040 0x7f3530073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f353743a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35301076f0 0x7f3530074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f353743a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35301a5c70 con 0x7f35301076f0 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f353743a700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35301a5de0 con 0x7f3530101680 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f353743a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35301a5fa0 con 0x7f3530102040 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35359d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35301076f0 0x7f3530074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35359d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35301076f0 0x7f3530074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59398/0 (socket says 172.21.15.27:59398) 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35349d5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3530102040 0x7f3530073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35359d7700 1 -- 172.21.15.27:0/3494859427 learned_addr learned my addr 172.21.15.27:0/3494859427 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35351d6700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3530101680 0x7f35300738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35359d7700 1 -- 172.21.15.27:0/3494859427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3530102040 msgr2=0x7f3530073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35359d7700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3530102040 0x7f3530073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35359d7700 1 -- 172.21.15.27:0/3494859427 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3530101680 msgr2=0x7f35300738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35359d7700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3530101680 0x7f35300738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.733+0000 7f35359d7700 1 -- 172.21.15.27:0/3494859427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528009d00 con 0x7f35301076f0 2023-12-17T12:09:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.734+0000 7f35359d7700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35301076f0 0x7f3530074350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f352800b070 tx=0x7f35280079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.734+0000 7f35267fc700 1 -- 172.21.15.27:0/3494859427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352800e040 con 0x7f35301076f0 2023-12-17T12:09:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.734+0000 7f35267fc700 1 -- 172.21.15.27:0/3494859427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3528009460 con 0x7f35301076f0 2023-12-17T12:09:03.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.734+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35301a6220 con 0x7f35301076f0 2023-12-17T12:09:03.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.734+0000 7f35267fc700 1 -- 172.21.15.27:0/3494859427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352801de80 con 0x7f35301076f0 2023-12-17T12:09:03.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.734+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35301a6690 con 0x7f35301076f0 2023-12-17T12:09:03.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.735+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3530109b80 con 0x7f35301076f0 2023-12-17T12:09:03.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.737+0000 7f35267fc700 1 -- 172.21.15.27:0/3494859427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f352801d480 con 0x7f35301076f0 2023-12-17T12:09:03.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.738+0000 7f35267fc700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f351c0a1590 0x7f351c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:03.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.739+0000 7f35267fc700 1 -- 172.21.15.27:0/3494859427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f35280ce5e0 con 0x7f35301076f0 2023-12-17T12:09:03.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.739+0000 7f35351d6700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f351c0a1590 0x7f351c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:03.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.739+0000 7f35351d6700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f351c0a1590 0x7f351c0a3a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f352c005ea0 tx=0x7f352c005e30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:03.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:03.742+0000 7f35267fc700 1 -- 172.21.15.27:0/3494859427 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3528097c20 con 0x7f35301076f0 2023-12-17T12:09:04.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.013+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3530074db0 con 0x7f35301076f0 2023-12-17T12:09:04.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.019+0000 7f35267fc700 1 -- 172.21.15.27:0/3494859427 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260403 (secure 0 0 0) 0x7f352809b240 con 0x7f35301076f0 2023-12-17T12:09:04.024 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:04.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f351c0a1590 msgr2=0x7f351c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f351c0a1590 0x7f351c0a3a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f352c005ea0 tx=0x7f352c005e30 comp rx=0 tx=0).stop 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35301076f0 msgr2=0x7f3530074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35301076f0 0x7f3530074350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f352800b070 tx=0x7f35280079b0 comp rx=0 tx=0).stop 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 shutdown_connections 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f351c0a1590 0x7f351c0a3a50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35301076f0 0x7f3530074350 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3530101680 0x7f35300738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 --2- 172.21.15.27:0/3494859427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3530102040 0x7f3530073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 >> 172.21.15.27:0/3494859427 conn(0x7f35300787c0 msgr2=0x7f3530100280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 shutdown_connections 2023-12-17T12:09:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:04.024+0000 7f353743a700 1 -- 172.21.15.27:0/3494859427 wait complete. 2023-12-17T12:09:04.436 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"active_gid":14154,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":46170456},{"type":"v1","addr":"172.21.15.27:6801","nonce":46170456}]},"active_addr":"172.21.15.27:6801/46170456","active_change":"2023-12-17T12:02:22.716776+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.27:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":441685550}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":2233895223}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":1532037128}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":971967861}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":2229567549}]}]} 2023-12-17T12:09:04.441 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-17T12:09:04.441 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-17T12:09:04.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-17T12:09:04.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:04 smithi027 ceph-mon[36871]: pgmap v272: 97 pgs: 97 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2023-12-17T12:09:04.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:04 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3494859427' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-17T12:09:04.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:04 smithi141 ceph-mon[41225]: pgmap v272: 97 pgs: 97 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2023-12-17T12:09:04.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:04 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3494859427' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-17T12:09:04.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:04 smithi192 ceph-mon[41899]: pgmap v272: 97 pgs: 97 active+clean; 622 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2023-12-17T12:09:04.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:04 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3494859427' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-17T12:09:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:06.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.096+0000 7f202b0f9700 1 -- 172.21.15.27:0/2287833798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2024101680 msgr2=0x7f2024101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:06.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.096+0000 7f202b0f9700 1 --2- 172.21.15.27:0/2287833798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2024101680 0x7f2024101b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2010009990 tx=0x7f2010009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.097+0000 7f202b0f9700 1 -- 172.21.15.27:0/2287833798 shutdown_connections 2023-12-17T12:09:06.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.097+0000 7f202b0f9700 1 --2- 172.21.15.27:0/2287833798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2024102040 0x7f202410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.097+0000 7f202b0f9700 1 --2- 172.21.15.27:0/2287833798 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f20241076f0 0x7f2024107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.097+0000 7f202b0f9700 1 --2- 172.21.15.27:0/2287833798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2024101680 0x7f2024101b00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.097+0000 7f202b0f9700 1 -- 172.21.15.27:0/2287833798 >> 172.21.15.27:0/2287833798 conn(0x7f20240787c0 msgr2=0x7f2024078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:06.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.097+0000 7f202b0f9700 1 -- 172.21.15.27:0/2287833798 shutdown_connections 2023-12-17T12:09:06.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.097+0000 7f202b0f9700 1 -- 172.21.15.27:0/2287833798 wait complete. 2023-12-17T12:09:06.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.098+0000 7f202b0f9700 1 Processor -- start 2023-12-17T12:09:06.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.098+0000 7f202b0f9700 1 -- start start 2023-12-17T12:09:06.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.098+0000 7f202b0f9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2024101680 0x7f202419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:06.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.098+0000 7f202b0f9700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2024102040 0x7f202419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:06.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.098+0000 7f202b0f9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20241076f0 0x7f2024199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:06.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f202b0f9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2024199ff0 con 0x7f20241076f0 2023-12-17T12:09:06.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f202b0f9700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202419a160 con 0x7f2024102040 2023-12-17T12:09:06.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f202b0f9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202419a320 con 0x7f2024101680 2023-12-17T12:09:06.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2029696700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20241076f0 0x7f2024199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:06.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2029696700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20241076f0 0x7f2024199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58490/0 (socket says 172.21.15.27:58490) 2023-12-17T12:09:06.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2029696700 1 -- 172.21.15.27:0/1572810020 learned_addr learned my addr 172.21.15.27:0/1572810020 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:06.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2023fff700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2024102040 0x7f202419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:06.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2028e95700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2024101680 0x7f202419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2029696700 1 -- 172.21.15.27:0/1572810020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2024101680 msgr2=0x7f202419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2029696700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2024101680 0x7f202419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2029696700 1 -- 172.21.15.27:0/1572810020 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2024102040 msgr2=0x7f202419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2029696700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2024102040 0x7f202419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.099+0000 7f2029696700 1 -- 172.21.15.27:0/1572810020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100095f0 con 0x7f20241076f0 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.100+0000 7f2029696700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20241076f0 0x7f2024199990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f201c007f10 tx=0x7f201c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.100+0000 7f2021ffb700 1 -- 172.21.15.27:0/1572810020 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c017070 con 0x7f20241076f0 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.100+0000 7f2021ffb700 1 -- 172.21.15.27:0/1572810020 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f201c00f040 con 0x7f20241076f0 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.100+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202419a600 con 0x7f20241076f0 2023-12-17T12:09:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.100+0000 7f2021ffb700 1 -- 172.21.15.27:0/1572810020 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c005710 con 0x7f20241076f0 2023-12-17T12:09:06.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.100+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20241a6840 con 0x7f20241076f0 2023-12-17T12:09:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.101+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2024102d10 con 0x7f20241076f0 2023-12-17T12:09:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.103+0000 7f2021ffb700 1 -- 172.21.15.27:0/1572810020 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f201c005870 con 0x7f20241076f0 2023-12-17T12:09:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.104+0000 7f2021ffb700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f20140a1510 0x7f20140a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.105+0000 7f2021ffb700 1 -- 172.21.15.27:0/1572810020 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f201c0cc540 con 0x7f20241076f0 2023-12-17T12:09:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.105+0000 7f2028e95700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f20140a1510 0x7f20140a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.105+0000 7f2028e95700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f20140a1510 0x7f20140a39d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2018006fd0 tx=0x7f2018008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:06.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.109+0000 7f2021ffb700 1 -- 172.21.15.27:0/1572810020 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f201c095af0 con 0x7f20241076f0 2023-12-17T12:09:06.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.319+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f202404f290 con 0x7f20241076f0 2023-12-17T12:09:06.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.321+0000 7f2021ffb700 1 -- 172.21.15.27:0/1572810020 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22130 (secure 0 0 0) 0x7f201c099110 con 0x7f20241076f0 2023-12-17T12:09:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:06.322 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:08:50.161605+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:09:06.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f20140a1510 msgr2=0x7f20140a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:06.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f20140a1510 0x7f20140a39d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2018006fd0 tx=0x7f2018008040 comp rx=0 tx=0).stop 2023-12-17T12:09:06.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20241076f0 msgr2=0x7f2024199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:06.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20241076f0 0x7f2024199990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f201c007f10 tx=0x7f201c00d3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 shutdown_connections 2023-12-17T12:09:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f20140a1510 0x7f20140a39d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2024101680 0x7f202419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20241076f0 0x7f2024199990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 --2- 172.21.15.27:0/1572810020 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2024102040 0x7f202419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.323+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 >> 172.21.15.27:0/1572810020 conn(0x7f20240787c0 msgr2=0x7f20241001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.324+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 shutdown_connections 2023-12-17T12:09:06.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:06.324+0000 7f202b0f9700 1 -- 172.21.15.27:0/1572810020 wait complete. 2023-12-17T12:09:06.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:06 smithi027 ceph-mon[36871]: pgmap v273: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-12-17T12:09:06.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:06 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1572810020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:06.756 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-17T12:09:06.756 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-17T12:09:06.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:06 smithi141 ceph-mon[41225]: pgmap v273: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-12-17T12:09:06.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:06 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1572810020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:06.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:06 smithi192 ceph-mon[41899]: pgmap v273: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-12-17T12:09:06.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:06 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1572810020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:07.189 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.548+0000 7f807fcb4700 1 -- 172.21.15.27:0/3799512585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078069930 msgr2=0x7f807810e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.548+0000 7f807fcb4700 1 --2- 172.21.15.27:0/3799512585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078069930 0x7f807810e420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f807000b0d0 tx=0x7f807000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.548+0000 7f807fcb4700 1 -- 172.21.15.27:0/3799512585 shutdown_connections 2023-12-17T12:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.548+0000 7f807fcb4700 1 --2- 172.21.15.27:0/3799512585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078069930 0x7f807810e420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.548+0000 7f807fcb4700 1 --2- 172.21.15.27:0/3799512585 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8078107620 0x7f8078107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.548+0000 7f807fcb4700 1 --2- 172.21.15.27:0/3799512585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8078068f70 0x7f80780693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.548+0000 7f807fcb4700 1 -- 172.21.15.27:0/3799512585 >> 172.21.15.27:0/3799512585 conn(0x7f8078076570 msgr2=0x7f8078076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:08.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.549+0000 7f807fcb4700 1 -- 172.21.15.27:0/3799512585 shutdown_connections 2023-12-17T12:09:08.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.549+0000 7f807fcb4700 1 -- 172.21.15.27:0/3799512585 wait complete. 2023-12-17T12:09:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807fcb4700 1 Processor -- start 2023-12-17T12:09:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807fcb4700 1 -- start start 2023-12-17T12:09:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807fcb4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078068f70 0x7f8078100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:08.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807fcb4700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8078069930 0x7f8078100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807fcb4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8078107620 0x7f8078101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807fcb4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8078102570 con 0x7f8078068f70 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807fcb4700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781aa3f0 con 0x7f8078069930 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807fcb4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781aa560 con 0x7f8078107620 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.550+0000 7f807da50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078068f70 0x7f8078100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807da50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078068f70 0x7f8078100600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58508/0 (socket says 172.21.15.27:58508) 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807da50700 1 -- 172.21.15.27:0/1468387721 learned_addr learned my addr 172.21.15.27:0/1468387721 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807d24f700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8078069930 0x7f8078100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807e251700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8078107620 0x7f8078101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807da50700 1 -- 172.21.15.27:0/1468387721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8078107620 msgr2=0x7f8078101080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807da50700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8078107620 0x7f8078101080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807da50700 1 -- 172.21.15.27:0/1468387721 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8078069930 msgr2=0x7f8078100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807da50700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8078069930 0x7f8078100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807da50700 1 -- 172.21.15.27:0/1468387721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8070009d00 con 0x7f8078068f70 2023-12-17T12:09:08.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807e251700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8078107620 0x7f8078101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f807da50700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078068f70 0x7f8078100600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f807400d950 tx=0x7f807400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.551+0000 7f806effd700 1 -- 172.21.15.27:0/1468387721 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074009ae0 con 0x7f8078068f70 2023-12-17T12:09:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.552+0000 7f806effd700 1 -- 172.21.15.27:0/1468387721 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8074009c40 con 0x7f8078068f70 2023-12-17T12:09:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.552+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80781aa840 con 0x7f8078068f70 2023-12-17T12:09:08.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.552+0000 7f806effd700 1 -- 172.21.15.27:0/1468387721 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400f620 con 0x7f8078068f70 2023-12-17T12:09:08.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.552+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80781aad90 con 0x7f8078068f70 2023-12-17T12:09:08.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.553+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8078109a20 con 0x7f8078068f70 2023-12-17T12:09:08.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.555+0000 7f806effd700 1 -- 172.21.15.27:0/1468387721 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f8074010460 con 0x7f8078068f70 2023-12-17T12:09:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.557+0000 7f806effd700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f80640a1590 0x7f80640a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.557+0000 7f806effd700 1 -- 172.21.15.27:0/1468387721 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f80740ccc30 con 0x7f8078068f70 2023-12-17T12:09:08.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.557+0000 7f807d24f700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f80640a1590 0x7f80640a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.558+0000 7f807d24f700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f80640a1590 0x7f80640a3a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8068005950 tx=0x7f80680058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:08.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.561+0000 7f806effd700 1 -- 172.21.15.27:0/1468387721 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f80740961e0 con 0x7f8078068f70 2023-12-17T12:09:08.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:08 smithi027 ceph-mon[36871]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-17T12:09:08.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:08 smithi141 ceph-mon[41225]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-17T12:09:08.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.770+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8078102cc0 con 0x7f8078068f70 2023-12-17T12:09:08.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.772+0000 7f806effd700 1 -- 172.21.15.27:0/1468387721 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22130 (secure 0 0 0) 0x7f8074020090 con 0x7f8078068f70 2023-12-17T12:09:08.773 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:08.774 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:08:50.161605+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f80640a1590 msgr2=0x7f80640a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f80640a1590 0x7f80640a3a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8068005950 tx=0x7f80680058e0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078068f70 msgr2=0x7f8078100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078068f70 0x7f8078100600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f807400d950 tx=0x7f807400dd10 comp rx=0 tx=0).stop 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 shutdown_connections 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f80640a1590 0x7f80640a3a50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8078107620 0x7f8078101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8078068f70 0x7f8078100600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 --2- 172.21.15.27:0/1468387721 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8078069930 0x7f8078100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.774+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 >> 172.21.15.27:0/1468387721 conn(0x7f8078076570 msgr2=0x7f80780ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.775+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 shutdown_connections 2023-12-17T12:09:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:08.775+0000 7f807fcb4700 1 -- 172.21.15.27:0/1468387721 wait complete. 2023-12-17T12:09:08.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:08 smithi192 ceph-mon[41899]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-17T12:09:09.181 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-17T12:09:09.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-17T12:09:09.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-17T12:09:09.183 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-17T12:09:09.183 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-17T12:09:09.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-17T12:09:09.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-17T12:09:09.185 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-17T12:09:09.185 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-12-17T12:09:09.186 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-12-17T12:09:09.186 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-12-17T12:09:09.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-12-17T12:09:09.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:09 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1468387721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:09.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:09 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1468387721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:09.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:09 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1468387721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:10.506 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:10.716 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:10.739 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:10 smithi027 ceph-mon[36871]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-17T12:09:10.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:10 smithi141 ceph-mon[41225]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-17T12:09:10.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:10 smithi192 ceph-mon[41899]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-17T12:09:12.036 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:12.139 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:12.142 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:12.305 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.309+0000 7f9c70aaa700 1 -- 172.21.15.27:0/1698024265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c100ef0 msgr2=0x7f9c6c10d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.309+0000 7f9c70aaa700 1 --2- 172.21.15.27:0/1698024265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c100ef0 0x7f9c6c10d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400b0d0 tx=0x7f9c6400b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.310+0000 7f9c70aaa700 1 -- 172.21.15.27:0/1698024265 shutdown_connections 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.310+0000 7f9c70aaa700 1 --2- 172.21.15.27:0/1698024265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c100ef0 0x7f9c6c10d330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.310+0000 7f9c70aaa700 1 --2- 172.21.15.27:0/1698024265 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9c6c106560 0x7f9c6c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.310+0000 7f9c70aaa700 1 --2- 172.21.15.27:0/1698024265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c100530 0x7f9c6c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.310+0000 7f9c70aaa700 1 -- 172.21.15.27:0/1698024265 >> 172.21.15.27:0/1698024265 conn(0x7f9c6c076570 msgr2=0x7f9c6c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.310+0000 7f9c70aaa700 1 -- 172.21.15.27:0/1698024265 shutdown_connections 2023-12-17T12:09:12.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.310+0000 7f9c70aaa700 1 -- 172.21.15.27:0/1698024265 wait complete. 2023-12-17T12:09:12.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.311+0000 7f9c70aaa700 1 Processor -- start 2023-12-17T12:09:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.311+0000 7f9c70aaa700 1 -- start start 2023-12-17T12:09:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c70aaa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c100530 0x7f9c6c107340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c70aaa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c100ef0 0x7f9c6c107880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c70aaa700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9c6c106560 0x7f9c6c107dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c70aaa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c1aa1f0 con 0x7f9c6c100530 2023-12-17T12:09:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c70aaa700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c1aa330 con 0x7f9c6c106560 2023-12-17T12:09:12.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c70aaa700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c1aa470 con 0x7f9c6c100ef0 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c69d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c100ef0 0x7f9c6c107880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c69d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c100ef0 0x7f9c6c107880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:41132/0 (socket says 172.21.15.27:41132) 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c69d9b700 1 -- 172.21.15.27:0/2779394166 learned_addr learned my addr 172.21.15.27:0/2779394166 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.312+0000 7f9c6ad9d700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9c6c106560 0x7f9c6c107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.313+0000 7f9c6ad9d700 1 -- 172.21.15.27:0/2779394166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c100ef0 msgr2=0x7f9c6c107880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.313+0000 7f9c6ad9d700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c100ef0 0x7f9c6c107880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.313+0000 7f9c6ad9d700 1 -- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c100530 msgr2=0x7f9c6c107340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.313+0000 7f9c6a59c700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c100530 0x7f9c6c107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.313+0000 7f9c6ad9d700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c100530 0x7f9c6c107340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.313+0000 7f9c6ad9d700 1 -- 172.21.15.27:0/2779394166 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c64009d00 con 0x7f9c6c106560 2023-12-17T12:09:12.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.314+0000 7f9c6a59c700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c100530 0x7f9c6c107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:12.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.314+0000 7f9c6ad9d700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9c6c106560 0x7f9c6c107dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400b770 tx=0x7f9c64007e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:12.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.315+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6400e040 con 0x7f9c6c106560 2023-12-17T12:09:12.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.315+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c6400cdc0 con 0x7f9c6c106560 2023-12-17T12:09:12.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.315+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6401dcd0 con 0x7f9c6c106560 2023-12-17T12:09:12.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.315+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c1aa6f0 con 0x7f9c6c106560 2023-12-17T12:09:12.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.315+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c1aab10 con 0x7f9c6c106560 2023-12-17T12:09:12.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.317+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c6c110540 con 0x7f9c6c106560 2023-12-17T12:09:12.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.320+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9c6401d480 con 0x7f9c6c106560 2023-12-17T12:09:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.323+0000 7f9c5b7fe700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c540a1610 0x7f9c540a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.323+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c640ce120 con 0x7f9c6c106560 2023-12-17T12:09:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.324+0000 7f9c6a59c700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c540a1610 0x7f9c540a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.324+0000 7f9c5b7fe700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] conn(0x7f9c540aadf0 0x7f9c540ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.324+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 --> [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c540ad8c0 con 0x7f9c540aadf0 2023-12-17T12:09:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.324+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c640d0740 con 0x7f9c6c106560 2023-12-17T12:09:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.324+0000 7f9c69d9b700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] conn(0x7f9c540aadf0 0x7f9c540ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.325+0000 7f9c6a59c700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c540a1610 0x7f9c540a3ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009be0 tx=0x7f9c5c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.325+0000 7f9c69d9b700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] conn(0x7f9c540aadf0 0x7f9c540ad210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.326+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 <== osd.2 v2:172.21.15.192:6800/3357629736 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9c540ad8c0 con 0x7f9c540aadf0 2023-12-17T12:09:12.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.352+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 --> [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c6c101bd0 con 0x7f9c540aadf0 2023-12-17T12:09:12.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.353+0000 7f9c5b7fe700 1 -- 172.21.15.27:0/2779394166 <== osd.2 v2:172.21.15.192:6800/3357629736 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9c6c101bd0 con 0x7f9c540aadf0 2023-12-17T12:09:12.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 >> [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] conn(0x7f9c540aadf0 msgr2=0x7f9c540ad210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:12.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] conn(0x7f9c540aadf0 0x7f9c540ad210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:12.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c540a1610 msgr2=0x7f9c540a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:12.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9c540a1610 0x7f9c540a3ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009be0 tx=0x7f9c5c009260 comp rx=0 tx=0).stop 2023-12-17T12:09:12.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9c6c106560 msgr2=0x7f9c6c107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:12.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 --2- 172.21.15.27:0/2779394166 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9c6c106560 0x7f9c6c107dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400b770 tx=0x7f9c64007e00 comp rx=0 tx=0).stop 2023-12-17T12:09:12.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c6ad9d700 1 -- 172.21.15.27:0/2779394166 reap_dead start 2023-12-17T12:09:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 shutdown_connections 2023-12-17T12:09:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 >> 172.21.15.27:0/2779394166 conn(0x7f9c6c076570 msgr2=0x7f9c6c0ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 shutdown_connections 2023-12-17T12:09:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:12.354+0000 7f9c70aaa700 1 -- 172.21.15.27:0/2779394166 wait complete. 2023-12-17T12:09:12.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:12 smithi027 ceph-mon[36871]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-12-17T12:09:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:12.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:12 smithi141 ceph-mon[41225]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-12-17T12:09:12.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:12 smithi192 ceph-mon[41899]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-12-17T12:09:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:13.154 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 -- 172.21.15.27:0/1146344399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 msgr2=0x7fab340741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 --2- 172.21.15.27:0/1146344399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 0x7fab340741c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab28009990 tx=0x7fab28009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 -- 172.21.15.27:0/1146344399 shutdown_connections 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 --2- 172.21.15.27:0/1146344399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab34075550 0x7fab34073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 --2- 172.21.15.27:0/1146344399 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab34112350 0x7fab34114780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 --2- 172.21.15.27:0/1146344399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 0x7fab340741c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 -- 172.21.15.27:0/1146344399 >> 172.21.15.27:0/1146344399 conn(0x7fab340787c0 msgr2=0x7fab34078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 -- 172.21.15.27:0/1146344399 shutdown_connections 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.388+0000 7fab38829700 1 -- 172.21.15.27:0/1146344399 wait complete. 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.389+0000 7fab38829700 1 Processor -- start 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.389+0000 7fab38829700 1 -- start start 2023-12-17T12:09:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab38829700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 0x7fab3419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab38829700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab34075550 0x7fab3419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab38829700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab34112350 0x7fab341a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab38829700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab341a2ed0 con 0x7fab34073d40 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab38829700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab341a3040 con 0x7fab34112350 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab38829700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab341a3200 con 0x7fab34075550 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab3259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 0x7fab3419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab3259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 0x7fab3419dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58550/0 (socket says 172.21.15.27:58550) 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab3259c700 1 -- 172.21.15.27:0/2999634172 learned_addr learned my addr 172.21.15.27:0/2999634172 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.390+0000 7fab31d9b700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab34075550 0x7fab3419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab32d9d700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab34112350 0x7fab341a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab3259c700 1 -- 172.21.15.27:0/2999634172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab34075550 msgr2=0x7fab3419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab3259c700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab34075550 0x7fab3419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab3259c700 1 -- 172.21.15.27:0/2999634172 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab34112350 msgr2=0x7fab341a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab3259c700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab34112350 0x7fab341a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab3259c700 1 -- 172.21.15.27:0/2999634172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab280095f0 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab3259c700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 0x7fab3419dfd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab2400d950 tx=0x7fab2400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24009940 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab24010460 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab341a34e0 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.391+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2400f620 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.392+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab341aad40 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.393+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab34117da0 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.396+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fab240105d0 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.398+0000 7fab237fe700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fab1c0a1510 0x7fab1c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.398+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fab240cd110 con 0x7fab34073d40 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.398+0000 7fab31d9b700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fab1c0a1510 0x7fab1c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.398+0000 7fab237fe700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] conn(0x7fab1c0aacf0 0x7fab1c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.398+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 --> [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab1c0ad7c0 con 0x7fab1c0aacf0 2023-12-17T12:09:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.398+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab240cd480 con 0x7fab34073d40 2023-12-17T12:09:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.399+0000 7fab32d9d700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] conn(0x7fab1c0aacf0 0x7fab1c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.399+0000 7fab31d9b700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fab1c0a1510 0x7fab1c0a39d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fab3419f530 tx=0x7fab28005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.399+0000 7fab32d9d700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] conn(0x7fab1c0aacf0 0x7fab1c0ad110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.400+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 <== osd.5 v2:172.21.15.192:6808/1558095856 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fab1c0ad7c0 con 0x7fab1c0aacf0 2023-12-17T12:09:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.420+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 --> [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab34074e90 con 0x7fab1c0aacf0 2023-12-17T12:09:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab237fe700 1 -- 172.21.15.27:0/2999634172 <== osd.5 v2:172.21.15.192:6808/1558095856 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fab34074e90 con 0x7fab1c0aacf0 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 >> [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] conn(0x7fab1c0aacf0 msgr2=0x7fab1c0ad110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab38829700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] conn(0x7fab1c0aacf0 0x7fab1c0ad110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fab1c0a1510 msgr2=0x7fab1c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab38829700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fab1c0a1510 0x7fab1c0a39d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fab3419f530 tx=0x7fab28005fb0 comp rx=0 tx=0).stop 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 msgr2=0x7fab3419dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab38829700 1 --2- 172.21.15.27:0/2999634172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab34073d40 0x7fab3419dfd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab2400d950 tx=0x7fab2400dc60 comp rx=0 tx=0).stop 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab32d9d700 1 -- 172.21.15.27:0/2999634172 reap_dead start 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 shutdown_connections 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.421+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 >> 172.21.15.27:0/2999634172 conn(0x7fab340787c0 msgr2=0x7fab34110f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.422+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 shutdown_connections 2023-12-17T12:09:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:13.422+0000 7fab38829700 1 -- 172.21.15.27:0/2999634172 wait complete. 2023-12-17T12:09:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:14.755 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:14 smithi027 ceph-mon[36871]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-12-17T12:09:14.758 INFO:teuthology.orchestra.run.smithi027.stdout:77309411385 2023-12-17T12:09:14.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-17T12:09:14.763 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:14 smithi141 ceph-mon[41225]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-12-17T12:09:14.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:14 smithi192 ceph-mon[41899]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-12-17T12:09:16.600 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:16 smithi027 ceph-mon[36871]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-17T12:09:16.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:16 smithi141 ceph-mon[41225]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-17T12:09:16.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:16 smithi192 ceph-mon[41899]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-17T12:09:16.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 -- 172.21.15.27:0/447479429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b810e030 msgr2=0x7f03b8110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:16.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 --2- 172.21.15.27:0/447479429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b810e030 0x7f03b8110420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f03b000b0a0 tx=0x7f03b000b3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:16.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 -- 172.21.15.27:0/447479429 shutdown_connections 2023-12-17T12:09:16.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 --2- 172.21.15.27:0/447479429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b810e030 0x7f03b8110420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:16.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 --2- 172.21.15.27:0/447479429 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f03b8073800 0x7f03b8073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 --2- 172.21.15.27:0/447479429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8074120 0x7f03b810daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 -- 172.21.15.27:0/447479429 >> 172.21.15.27:0/447479429 conn(0x7f03b80787c0 msgr2=0x7f03b8078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 -- 172.21.15.27:0/447479429 shutdown_connections 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.449+0000 7f03bd072700 1 -- 172.21.15.27:0/447479429 wait complete. 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.451+0000 7f03bd072700 1 Processor -- start 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.451+0000 7f03bd072700 1 -- start start 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03bd072700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b8073800 0x7f03b819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03bd072700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8074120 0x7f03b819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03bd072700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f03b810e030 0x7f03b8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03bd072700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b819a150 con 0x7f03b8073800 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03bd072700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b819a2c0 con 0x7f03b810e030 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03bd072700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b819a480 con 0x7f03b8074120 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b8073800 0x7f03b819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b8073800 0x7f03b819f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35450/0 (socket says 172.21.15.27:35450) 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b6d9d700 1 -- 172.21.15.27:0/3271194699 learned_addr learned my addr 172.21.15.27:0/3271194699 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b659c700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8074120 0x7f03b819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b6d9d700 1 -- 172.21.15.27:0/3271194699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8074120 msgr2=0x7f03b819fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b6d9d700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8074120 0x7f03b819fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b6d9d700 1 -- 172.21.15.27:0/3271194699 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f03b810e030 msgr2=0x7f03b8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b6d9d700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f03b810e030 0x7f03b8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b6d9d700 1 -- 172.21.15.27:0/3271194699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b0009d00 con 0x7f03b8073800 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.452+0000 7f03b659c700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b8074120 0x7f03b819fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.453+0000 7f03b6d9d700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b8073800 0x7f03b819f9b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03a800e9a0 tx=0x7f03a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.453+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8009790 con 0x7f03b8073800 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.453+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03a800cbd0 con 0x7f03b8073800 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.453+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b819a760 con 0x7f03b8073800 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.453+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8018680 con 0x7f03b8073800 2023-12-17T12:09:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.453+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b81a69a0 con 0x7f03b8073800 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.454+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03b8113910 con 0x7f03b8073800 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.456+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f03a800cd40 con 0x7f03b8073800 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.459+0000 7f039ffff700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f03a00a1590 0x7f03a00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.459+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f03a8014070 con 0x7f03b8073800 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.459+0000 7f03b659c700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f03a00a1590 0x7f03a00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.460+0000 7f039ffff700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] conn(0x7f03a00aad70 0x7f03a00ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.460+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 --> [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a00ad840 con 0x7f03a00aad70 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.460+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03a80cd660 con 0x7f03b8073800 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.460+0000 7f03b659c700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f03a00a1590 0x7f03a00a3a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03b819acb0 tx=0x7f03ac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.460+0000 7f03b759e700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] conn(0x7f03a00aad70 0x7f03a00ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.461+0000 7f03b759e700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] conn(0x7f03a00aad70 0x7f03a00ad190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.462+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 <== osd.4 v2:172.21.15.141:6808/2292839453 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f03a00ad840 con 0x7f03a00aad70 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.480+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 --> [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f03b8061c50 con 0x7f03a00aad70 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.481+0000 7f039ffff700 1 -- 172.21.15.27:0/3271194699 <== osd.4 v2:172.21.15.141:6808/2292839453 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f03b8061c50 con 0x7f03a00aad70 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 >> [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] conn(0x7f03a00aad70 msgr2=0x7f03a00ad190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] conn(0x7f03a00aad70 0x7f03a00ad190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f03a00a1590 msgr2=0x7f03a00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f03a00a1590 0x7f03a00a3a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03b819acb0 tx=0x7f03ac009260 comp rx=0 tx=0).stop 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b8073800 msgr2=0x7f03b819f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 --2- 172.21.15.27:0/3271194699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b8073800 0x7f03b819f9b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03a800e9a0 tx=0x7f03a800ed60 comp rx=0 tx=0).stop 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03b759e700 1 -- 172.21.15.27:0/3271194699 reap_dead start 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 shutdown_connections 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 >> 172.21.15.27:0/3271194699 conn(0x7f03b80787c0 msgr2=0x7f03b80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 shutdown_connections 2023-12-17T12:09:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:16.482+0000 7f03bd072700 1 -- 172.21.15.27:0/3271194699 wait complete. 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.198+0000 7f01cff69700 1 -- 172.21.15.27:0/1744862530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8102040 msgr2=0x7f01c810e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.198+0000 7f01cff69700 1 --2- 172.21.15.27:0/1744862530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8102040 0x7f01c810e430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f01c000b0a0 tx=0x7f01c000b3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.199+0000 7f01cff69700 1 -- 172.21.15.27:0/1744862530 shutdown_connections 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.199+0000 7f01cff69700 1 --2- 172.21.15.27:0/1744862530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8102040 0x7f01c810e430 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.199+0000 7f01cff69700 1 --2- 172.21.15.27:0/1744862530 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f01c81076f0 0x7f01c8107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.199+0000 7f01cff69700 1 --2- 172.21.15.27:0/1744862530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c8101680 0x7f01c8101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.199+0000 7f01cff69700 1 -- 172.21.15.27:0/1744862530 >> 172.21.15.27:0/1744862530 conn(0x7f01c80787c0 msgr2=0x7f01c8078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.199+0000 7f01cff69700 1 -- 172.21.15.27:0/1744862530 shutdown_connections 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.199+0000 7f01cff69700 1 -- 172.21.15.27:0/1744862530 wait complete. 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.200+0000 7f01cff69700 1 Processor -- start 2023-12-17T12:09:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.200+0000 7f01cff69700 1 -- start start 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cff69700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8101680 0x7f01c80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cff69700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c8102040 0x7f01c8073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cff69700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f01c81076f0 0x7f01c8074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cff69700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c81a5e10 con 0x7f01c8101680 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cff69700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c81a5f80 con 0x7f01c81076f0 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cff69700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c81a6140 con 0x7f01c8102040 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cdd05700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8101680 0x7f01c80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cdd05700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8101680 0x7f01c80738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35470/0 (socket says 172.21.15.27:35470) 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cdd05700 1 -- 172.21.15.27:0/3101902406 learned_addr learned my addr 172.21.15.27:0/3101902406 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.201+0000 7f01cd504700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c8102040 0x7f01c8073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01cdd05700 1 -- 172.21.15.27:0/3101902406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c8102040 msgr2=0x7f01c8073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01cdd05700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c8102040 0x7f01c8073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01cdd05700 1 -- 172.21.15.27:0/3101902406 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f01c81076f0 msgr2=0x7f01c8074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01cdd05700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f01c81076f0 0x7f01c8074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01cdd05700 1 -- 172.21.15.27:0/3101902406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c0009d00 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01cdd05700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8101680 0x7f01c80738d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01c400e9a0 tx=0x7f01c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4009790 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c81a6420 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01c400cbd0 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4018680 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.202+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c81a6970 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.203+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01c8111640 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.208+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f01c400cd40 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.209+0000 7f01beffd700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f01b40a1510 0x7f01b40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.209+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f01c4014070 con 0x7f01c8101680 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.210+0000 7f01cd504700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f01b40a1510 0x7f01b40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.210+0000 7f01beffd700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] conn(0x7f01b40aacf0 0x7f01b40ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.210+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 --> [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f01b40ad7c0 con 0x7f01b40aacf0 2023-12-17T12:09:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.210+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01c40cd7e0 con 0x7f01c8101680 2023-12-17T12:09:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.210+0000 7f01cd504700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f01b40a1510 0x7f01b40a39d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01c8075000 tx=0x7f01b800b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.210+0000 7f01ce506700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] conn(0x7f01b40aacf0 0x7f01b40ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.211+0000 7f01ce506700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] conn(0x7f01b40aacf0 0x7f01b40ad110 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.212+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 <== osd.11 v2:172.21.15.192:6824/2290390540 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f01b40ad7c0 con 0x7f01b40aacf0 2023-12-17T12:09:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.230+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 --> [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f01c8102d10 con 0x7f01b40aacf0 2023-12-17T12:09:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.230+0000 7f01beffd700 1 -- 172.21.15.27:0/3101902406 <== osd.11 v2:172.21.15.192:6824/2290390540 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f01c8102d10 con 0x7f01b40aacf0 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 >> [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] conn(0x7f01b40aacf0 msgr2=0x7f01b40ad110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] conn(0x7f01b40aacf0 0x7f01b40ad110 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f01b40a1510 msgr2=0x7f01b40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f01b40a1510 0x7f01b40a39d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01c8075000 tx=0x7f01b800b410 comp rx=0 tx=0).stop 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8101680 msgr2=0x7f01c80738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 --2- 172.21.15.27:0/3101902406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c8101680 0x7f01c80738d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01c400e9a0 tx=0x7f01c400ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01ce506700 1 -- 172.21.15.27:0/3101902406 reap_dead start 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 shutdown_connections 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 >> 172.21.15.27:0/3101902406 conn(0x7f01c80787c0 msgr2=0x7f01c8100370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 shutdown_connections 2023-12-17T12:09:17.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:17.231+0000 7f01cff69700 1 -- 172.21.15.27:0/3101902406 wait complete. 2023-12-17T12:09:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.217+0000 7fd81b1b1700 1 -- 172.21.15.27:0/3977165441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814075550 msgr2=0x7fd814073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.217+0000 7fd81b1b1700 1 --2- 172.21.15.27:0/3977165441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814075550 0x7fd814073800 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd808009990 tx=0x7fd808009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.217+0000 7fd81b1b1700 1 -- 172.21.15.27:0/3977165441 shutdown_connections 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.217+0000 7fd81b1b1700 1 --2- 172.21.15.27:0/3977165441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd814112350 0x7fd814114780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.217+0000 7fd81b1b1700 1 --2- 172.21.15.27:0/3977165441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814075550 0x7fd814073800 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.217+0000 7fd81b1b1700 1 --2- 172.21.15.27:0/3977165441 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd814073d40 0x7fd8140741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.217+0000 7fd81b1b1700 1 -- 172.21.15.27:0/3977165441 >> 172.21.15.27:0/3977165441 conn(0x7fd8140787c0 msgr2=0x7fd814078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.217+0000 7fd81b1b1700 1 -- 172.21.15.27:0/3977165441 shutdown_connections 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.218+0000 7fd81b1b1700 1 -- 172.21.15.27:0/3977165441 wait complete. 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.218+0000 7fd81b1b1700 1 Processor -- start 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81b1b1700 1 -- start start 2023-12-17T12:09:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81b1b1700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd814073d40 0x7fd81419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81b1b1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd814075550 0x7fd81419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81b1b1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814112350 0x7fd8141a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81b1b1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a2f60 con 0x7fd814112350 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81b1b1700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a30d0 con 0x7fd814073d40 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81b1b1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a3290 con 0x7fd814075550 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81974e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814112350 0x7fd8141a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81974e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814112350 0x7fd8141a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35494/0 (socket says 172.21.15.27:35494) 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd81974e700 1 -- 172.21.15.27:0/339512601 learned_addr learned my addr 172.21.15.27:0/339512601 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.219+0000 7fd818f4d700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd814073d40 0x7fd81419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd813fff700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd814075550 0x7fd81419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd81974e700 1 -- 172.21.15.27:0/339512601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd814075550 msgr2=0x7fd81419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd81974e700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd814075550 0x7fd81419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd81974e700 1 -- 172.21.15.27:0/339512601 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd814073d40 msgr2=0x7fd81419e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd81974e700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd814073d40 0x7fd81419e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd81974e700 1 -- 172.21.15.27:0/339512601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080095f0 con 0x7fd814112350 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd81974e700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814112350 0x7fd8141a2950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd80c007ee0 tx=0x7fd80c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00ddd0 con 0x7fd814112350 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8141a3570 con 0x7fd814112350 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd80c00f040 con 0x7fd814112350 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.220+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c005970 con 0x7fd814112350 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.221+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8141aadd0 con 0x7fd814112350 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.222+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd814117dd0 con 0x7fd814112350 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.224+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fd80c004fb0 con 0x7fd814112350 2023-12-17T12:09:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.225+0000 7fd811ffb700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd8040a1510 0x7fd8040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.225+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd80c0cc600 con 0x7fd814112350 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.225+0000 7fd818f4d700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd8040a1510 0x7fd8040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.226+0000 7fd811ffb700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] conn(0x7fd8040aacf0 0x7fd8040ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.226+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 --> [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8040ad7c0 con 0x7fd8040aacf0 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.226+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd80c0cebe0 con 0x7fd814112350 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.226+0000 7fd813fff700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] conn(0x7fd8040aacf0 0x7fd8040ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.226+0000 7fd818f4d700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd8040a1510 0x7fd8040a39d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd80800b5c0 tx=0x7fd808005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.227+0000 7fd813fff700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] conn(0x7fd8040aacf0 0x7fd8040ad110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.227+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 <== osd.3 v2:172.21.15.27:6810/130987497 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd8040ad7c0 con 0x7fd8040aacf0 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.245+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 --> [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd814074e90 con 0x7fd8040aacf0 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.245+0000 7fd811ffb700 1 -- 172.21.15.27:0/339512601 <== osd.3 v2:172.21.15.27:6810/130987497 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd814074e90 con 0x7fd8040aacf0 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] conn(0x7fd8040aacf0 msgr2=0x7fd8040ad110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] conn(0x7fd8040aacf0 0x7fd8040ad110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd8040a1510 msgr2=0x7fd8040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fd8040a1510 0x7fd8040a39d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd80800b5c0 tx=0x7fd808005fb0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814112350 msgr2=0x7fd8141a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 --2- 172.21.15.27:0/339512601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd814112350 0x7fd8141a2950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd80c007ee0 tx=0x7fd80c00d3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81974e700 1 -- 172.21.15.27:0/339512601 reap_dead start 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 shutdown_connections 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 >> 172.21.15.27:0/339512601 conn(0x7fd8140787c0 msgr2=0x7fd814111080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 shutdown_connections 2023-12-17T12:09:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:18.246+0000 7fd81b1b1700 1 -- 172.21.15.27:0/339512601 wait complete. 2023-12-17T12:09:18.706 INFO:teuthology.orchestra.run.smithi027.stdout:158913789996 2023-12-17T12:09:18.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-17T12:09:18.766 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:18 smithi141 ceph-mon[41225]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-17T12:09:18.771 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:18 smithi027 ceph-mon[36871]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-17T12:09:18.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:18 smithi192 ceph-mon[41899]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-17T12:09:18.876 INFO:teuthology.orchestra.run.smithi027.stdout:330712481809 2023-12-17T12:09:18.876 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-12-17T12:09:19.151 INFO:teuthology.orchestra.run.smithi027.stdout:137438953521 2023-12-17T12:09:19.152 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-17T12:09:19.277 INFO:teuthology.orchestra.run.smithi027.stdout:107374182453 2023-12-17T12:09:19.277 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-17T12:09:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.470+0000 7f05ab4d3700 1 -- 172.21.15.27:0/2843258075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a4107620 msgr2=0x7f05a4107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.470+0000 7f05ab4d3700 1 --2- 172.21.15.27:0/2843258075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a4107620 0x7f05a4107a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f05980099c0 tx=0x7f0598009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.471+0000 7f05ab4d3700 1 -- 172.21.15.27:0/2843258075 shutdown_connections 2023-12-17T12:09:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.471+0000 7f05ab4d3700 1 --2- 172.21.15.27:0/2843258075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a4069930 0x7f05a410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.471+0000 7f05ab4d3700 1 --2- 172.21.15.27:0/2843258075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a4107620 0x7f05a4107a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.471+0000 7f05ab4d3700 1 --2- 172.21.15.27:0/2843258075 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f05a4068f70 0x7f05a40693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.471+0000 7f05ab4d3700 1 -- 172.21.15.27:0/2843258075 >> 172.21.15.27:0/2843258075 conn(0x7f05a4076570 msgr2=0x7f05a4076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.471+0000 7f05ab4d3700 1 -- 172.21.15.27:0/2843258075 shutdown_connections 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.471+0000 7f05ab4d3700 1 -- 172.21.15.27:0/2843258075 wait complete. 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.473+0000 7f05ab4d3700 1 Processor -- start 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.473+0000 7f05ab4d3700 1 -- start start 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05ab4d3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a4068f70 0x7f05a410a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05ab4d3700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f05a4069930 0x7f05a410ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05ab4d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a4107620 0x7f05a410b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05ab4d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41aa140 con 0x7f05a4107620 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05ab4d3700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41aa2b0 con 0x7f05a4069930 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05ab4d3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41aa470 con 0x7f05a4068f70 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05a926f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a4068f70 0x7f05a410a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05a9a70700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a4107620 0x7f05a410b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05a926f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a4068f70 0x7f05a410a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:40290/0 (socket says 172.21.15.27:40290) 2023-12-17T12:09:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.474+0000 7f05a926f700 1 -- 172.21.15.27:0/1716188137 learned_addr learned my addr 172.21.15.27:0/1716188137 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.475+0000 7f05a9a70700 1 -- 172.21.15.27:0/1716188137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a4068f70 msgr2=0x7f05a410a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.475+0000 7f05a9a70700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a4068f70 0x7f05a410a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.475+0000 7f05a9a70700 1 -- 172.21.15.27:0/1716188137 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f05a4069930 msgr2=0x7f05a410ad70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.475+0000 7f05a9a70700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f05a4069930 0x7f05a410ad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.475+0000 7f05a9a70700 1 -- 172.21.15.27:0/1716188137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980095f0 con 0x7f05a4107620 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.475+0000 7f05a926f700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a4068f70 0x7f05a410a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.475+0000 7f05a9a70700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a4107620 0x7f05a410b2b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f059c00efc0 tx=0x7f059c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.476+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c018070 con 0x7f05a4107620 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.476+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f059c00f040 con 0x7f05a4107620 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.476+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c00cc70 con 0x7f05a4107620 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.476+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a41aa6f0 con 0x7f05a4107620 2023-12-17T12:09:19.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.476+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a41aaba0 con 0x7f05a4107620 2023-12-17T12:09:19.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.477+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05a41015d0 con 0x7f05a4107620 2023-12-17T12:09:19.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.481+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f059c003a00 con 0x7f05a4107620 2023-12-17T12:09:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.483+0000 7f05967fc700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f05900a1660 0x7f05900a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.483+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f059c0cd3c0 con 0x7f05a4107620 2023-12-17T12:09:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.483+0000 7f05967fc700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] conn(0x7f05900aae40 0x7f05900ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.483+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 --> [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05900ad910 con 0x7f05900aae40 2023-12-17T12:09:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.483+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f059c0cd860 con 0x7f05a4107620 2023-12-17T12:09:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.483+0000 7f05a926f700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f05900a1660 0x7f05900a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.484+0000 7f05a8a6e700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] conn(0x7f05900aae40 0x7f05900ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.484+0000 7f05a926f700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f05900a1660 0x7f05900a3b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0598009fd0 tx=0x7f0598005ab0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.485+0000 7f05a8a6e700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] conn(0x7f05900aae40 0x7f05900ad260 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.487+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 <== osd.0 v2:172.21.15.27:6802/4043808604 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f05900ad910 con 0x7f05900aae40 2023-12-17T12:09:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.510+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 --> [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f05a4106ab0 con 0x7f05900aae40 2023-12-17T12:09:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.510+0000 7f05967fc700 1 -- 172.21.15.27:0/1716188137 <== osd.0 v2:172.21.15.27:6802/4043808604 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f05a4106ab0 con 0x7f05900aae40 2023-12-17T12:09:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.511+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] conn(0x7f05900aae40 msgr2=0x7f05900ad260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.511+0000 7f05ab4d3700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] conn(0x7f05900aae40 0x7f05900ad260 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.511+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f05900a1660 msgr2=0x7f05900a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.511+0000 7f05ab4d3700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f05900a1660 0x7f05900a3b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0598009fd0 tx=0x7f0598005ab0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.511+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a4107620 msgr2=0x7f05a410b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.511+0000 7f05ab4d3700 1 --2- 172.21.15.27:0/1716188137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a4107620 0x7f05a410b2b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f059c00efc0 tx=0x7f059c00c3a0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.511+0000 7f05a9a70700 1 -- 172.21.15.27:0/1716188137 reap_dead start 2023-12-17T12:09:19.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.512+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 shutdown_connections 2023-12-17T12:09:19.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.512+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 >> 172.21.15.27:0/1716188137 conn(0x7f05a4076570 msgr2=0x7f05a4105150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:19.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.512+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 shutdown_connections 2023-12-17T12:09:19.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.512+0000 7f05ab4d3700 1 -- 172.21.15.27:0/1716188137 wait complete. 2023-12-17T12:09:19.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 -- 172.21.15.27:0/523616412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9581028b0 msgr2=0x7fe95810d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 --2- 172.21.15.27:0/523616412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9581028b0 0x7fe95810d290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe95000b0d0 tx=0x7fe95000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 -- 172.21.15.27:0/523616412 shutdown_connections 2023-12-17T12:09:19.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 --2- 172.21.15.27:0/523616412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9581028b0 0x7fe95810d290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 --2- 172.21.15.27:0/523616412 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe958100600 0x7fe9581009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 --2- 172.21.15.27:0/523616412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe958101ef0 0x7fe958102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 -- 172.21.15.27:0/523616412 >> 172.21.15.27:0/523616412 conn(0x7fe9580fbe10 msgr2=0x7fe9580fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:19.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 -- 172.21.15.27:0/523616412 shutdown_connections 2023-12-17T12:09:19.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.594+0000 7fe95d8ac700 1 -- 172.21.15.27:0/523616412 wait complete. 2023-12-17T12:09:19.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.596+0000 7fe95d8ac700 1 Processor -- start 2023-12-17T12:09:19.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.596+0000 7fe95d8ac700 1 -- start start 2023-12-17T12:09:19.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95d8ac700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe958100600 0x7fe95810dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95d8ac700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe958101ef0 0x7fe95810e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95d8ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9581028b0 0x7fe95810e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95d8ac700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9581a5bf0 con 0x7fe958100600 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95d8ac700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9581a5d60 con 0x7fe958101ef0 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95d8ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9581a5f20 con 0x7fe9581028b0 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95c8aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe958100600 0x7fe95810dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95c8aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe958100600 0x7fe95810dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35534/0 (socket says 172.21.15.27:35534) 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe95c8aa700 1 -- 172.21.15.27:0/4022232750 learned_addr learned my addr 172.21.15.27:0/4022232750 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.597+0000 7fe957fff700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe958101ef0 0x7fe95810e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95d0ab700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9581028b0 0x7fe95810e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95c8aa700 1 -- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9581028b0 msgr2=0x7fe95810e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95c8aa700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9581028b0 0x7fe95810e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95c8aa700 1 -- 172.21.15.27:0/4022232750 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe958101ef0 msgr2=0x7fe95810e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95c8aa700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe958101ef0 0x7fe95810e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95c8aa700 1 -- 172.21.15.27:0/4022232750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009d00 con 0x7fe958100600 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95d0ab700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9581028b0 0x7fe95810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95c8aa700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe958100600 0x7fe95810dc60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00e9a0 tx=0x7fe94c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c009790 con 0x7fe958100600 2023-12-17T12:09:19.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe94c00cbd0 con 0x7fe958100600 2023-12-17T12:09:19.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9581a6200 con 0x7fe958100600 2023-12-17T12:09:19.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.598+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c018680 con 0x7fe958100600 2023-12-17T12:09:19.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.599+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9581a6750 con 0x7fe958100600 2023-12-17T12:09:19.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.600+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe9581115c0 con 0x7fe958100600 2023-12-17T12:09:19.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.602+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe94c00cd40 con 0x7fe958100600 2023-12-17T12:09:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.603+0000 7fe955ffb700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe9480a1590 0x7fe9480a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.604+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe94c014070 con 0x7fe958100600 2023-12-17T12:09:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.604+0000 7fe957fff700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe9480a1590 0x7fe9480a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.604+0000 7fe955ffb700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] conn(0x7fe9480aad70 0x7fe9480ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.604+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 --> [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9480ad840 con 0x7fe9480aad70 2023-12-17T12:09:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.604+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe94c0cd7a0 con 0x7fe958100600 2023-12-17T12:09:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.605+0000 7fe957fff700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe9480a1590 0x7fe9480a3a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe95810f390 tx=0x7fe944009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.605+0000 7fe95d0ab700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] conn(0x7fe9480aad70 0x7fe9480ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.605+0000 7fe95d0ab700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] conn(0x7fe9480aad70 0x7fe9480ad190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.606+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 <== osd.8 v2:172.21.15.192:6816/1098230082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe9480ad840 con 0x7fe9480aad70 2023-12-17T12:09:19.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.625+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 --> [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe958101a00 con 0x7fe9480aad70 2023-12-17T12:09:19.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.626+0000 7fe955ffb700 1 -- 172.21.15.27:0/4022232750 <== osd.8 v2:172.21.15.192:6816/1098230082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe958101a00 con 0x7fe9480aad70 2023-12-17T12:09:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.628+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] conn(0x7fe9480aad70 msgr2=0x7fe9480ad190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.628+0000 7fe95d8ac700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] conn(0x7fe9480aad70 0x7fe9480ad190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.628+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe9480a1590 msgr2=0x7fe9480a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.628+0000 7fe95d8ac700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe9480a1590 0x7fe9480a3a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe95810f390 tx=0x7fe944009260 comp rx=0 tx=0).stop 2023-12-17T12:09:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.628+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe958100600 msgr2=0x7fe95810dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.628+0000 7fe95d8ac700 1 --2- 172.21.15.27:0/4022232750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe958100600 0x7fe95810dc60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00e9a0 tx=0x7fe94c00ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.629+0000 7fe95d0ab700 1 -- 172.21.15.27:0/4022232750 reap_dead start 2023-12-17T12:09:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.629+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 shutdown_connections 2023-12-17T12:09:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.629+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 >> 172.21.15.27:0/4022232750 conn(0x7fe9580fbe10 msgr2=0x7fe9580fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.629+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 shutdown_connections 2023-12-17T12:09:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.629+0000 7fe95d8ac700 1 -- 172.21.15.27:0/4022232750 wait complete. 2023-12-17T12:09:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.703+0000 7f7389a7d700 1 -- 172.21.15.27:0/2332748341 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7384068f70 msgr2=0x7f73840693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.703+0000 7f7389a7d700 1 --2- 172.21.15.27:0/2332748341 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7384068f70 0x7f73840693f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f73780099c0 tx=0x7f7378009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.704+0000 7f7389a7d700 1 -- 172.21.15.27:0/2332748341 shutdown_connections 2023-12-17T12:09:19.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.704+0000 7f7389a7d700 1 --2- 172.21.15.27:0/2332748341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7384107620 0x7f7384107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.704+0000 7f7389a7d700 1 --2- 172.21.15.27:0/2332748341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7384069930 0x7f738410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.704+0000 7f7389a7d700 1 --2- 172.21.15.27:0/2332748341 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7384068f70 0x7f73840693f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.704+0000 7f7389a7d700 1 -- 172.21.15.27:0/2332748341 >> 172.21.15.27:0/2332748341 conn(0x7f7384076570 msgr2=0x7f7384076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:19.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.704+0000 7f7389a7d700 1 -- 172.21.15.27:0/2332748341 shutdown_connections 2023-12-17T12:09:19.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.704+0000 7f7389a7d700 1 -- 172.21.15.27:0/2332748341 wait complete. 2023-12-17T12:09:19.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.705+0000 7f7389a7d700 1 Processor -- start 2023-12-17T12:09:19.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.705+0000 7f7389a7d700 1 -- start start 2023-12-17T12:09:19.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.706+0000 7f7389a7d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7384068f70 0x7f738419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.706+0000 7f7389a7d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7384069930 0x7f738419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.706+0000 7f7389a7d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7384107620 0x7f73841977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.706+0000 7f7389a7d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7384197e30 con 0x7f7384107620 2023-12-17T12:09:19.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.706+0000 7f7389a7d700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7384197fa0 con 0x7f7384068f70 2023-12-17T12:09:19.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.706+0000 7f7389a7d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7384198160 con 0x7f7384069930 2023-12-17T12:09:19.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7383fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7384107620 0x7f73841977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7383fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7384107620 0x7f73841977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35552/0 (socket says 172.21.15.27:35552) 2023-12-17T12:09:19.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7383fff700 1 -- 172.21.15.27:0/3295138162 learned_addr learned my addr 172.21.15.27:0/3295138162 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7382ffd700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7384069930 0x7f738419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f73837fe700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7384068f70 0x7f738419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7382ffd700 1 -- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7384068f70 msgr2=0x7f738419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7382ffd700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7384068f70 0x7f738419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7382ffd700 1 -- 172.21.15.27:0/3295138162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7384107620 msgr2=0x7f73841977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7382ffd700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7384107620 0x7f73841977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.707+0000 7f7382ffd700 1 -- 172.21.15.27:0/3295138162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73780095f0 con 0x7f7384069930 2023-12-17T12:09:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.708+0000 7f73837fe700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7384068f70 0x7f738419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.708+0000 7f7382ffd700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7384069930 0x7f738419dbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f73780098e0 tx=0x7f7378005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.708+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737801d070 con 0x7f7384069930 2023-12-17T12:09:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.708+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7378005670 con 0x7f7384069930 2023-12-17T12:09:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.708+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73841983e0 con 0x7f7384069930 2023-12-17T12:09:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.709+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737800f960 con 0x7f7384069930 2023-12-17T12:09:19.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.709+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7384100700 con 0x7f7384069930 2023-12-17T12:09:19.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.710+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f73841116c0 con 0x7f7384069930 2023-12-17T12:09:19.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.713+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7378004c30 con 0x7f7384069930 2023-12-17T12:09:19.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.715+0000 7f7380ff9700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f736c0a1510 0x7f736c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.715+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73780cd960 con 0x7f7384069930 2023-12-17T12:09:19.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.715+0000 7f7380ff9700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] conn(0x7f736c0aacf0 0x7f736c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:19.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.715+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 --> [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f736c0ad7c0 con 0x7f736c0aacf0 2023-12-17T12:09:19.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.715+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73780cde00 con 0x7f7384069930 2023-12-17T12:09:19.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.716+0000 7f7383fff700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] conn(0x7f736c0aacf0 0x7f736c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.716+0000 7f7383fff700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] conn(0x7f736c0aacf0 0x7f736c0ad110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.717+0000 7f73837fe700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f736c0a1510 0x7f736c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:19.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.717+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 <== osd.10 v2:172.21.15.141:6824/2486275006 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f736c0ad7c0 con 0x7f736c0aacf0 2023-12-17T12:09:19.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.718+0000 7f73837fe700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f736c0a1510 0x7f736c0a39d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7374009fd0 tx=0x7f7374009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.738+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 --> [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7384061cd0 con 0x7f736c0aacf0 2023-12-17T12:09:19.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.739+0000 7f7380ff9700 1 -- 172.21.15.27:0/3295138162 <== osd.10 v2:172.21.15.141:6824/2486275006 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7384061cd0 con 0x7f736c0aacf0 2023-12-17T12:09:19.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.739+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] conn(0x7f736c0aacf0 msgr2=0x7f736c0ad110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.739+0000 7f7389a7d700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] conn(0x7f736c0aacf0 0x7f736c0ad110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:19.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.739+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f736c0a1510 msgr2=0x7f736c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.739+0000 7f7389a7d700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f736c0a1510 0x7f736c0a39d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7374009fd0 tx=0x7f7374009260 comp rx=0 tx=0).stop 2023-12-17T12:09:19.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.739+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7384069930 msgr2=0x7f738419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:19.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.739+0000 7f7389a7d700 1 --2- 172.21.15.27:0/3295138162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7384069930 0x7f738419dbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f73780098e0 tx=0x7f7378005270 comp rx=0 tx=0).stop 2023-12-17T12:09:19.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.739+0000 7f7383fff700 1 -- 172.21.15.27:0/3295138162 reap_dead start 2023-12-17T12:09:19.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.740+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 shutdown_connections 2023-12-17T12:09:19.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.740+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 >> 172.21.15.27:0/3295138162 conn(0x7f7384076570 msgr2=0x7f73840ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:19.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.740+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 shutdown_connections 2023-12-17T12:09:19.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.740+0000 7f7389a7d700 1 -- 172.21.15.27:0/3295138162 wait complete. 2023-12-17T12:09:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.835+0000 7fee5d67b700 1 -- 172.21.15.27:0/1746522815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58101680 msgr2=0x7fee58101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.835+0000 7fee5d67b700 1 --2- 172.21.15.27:0/1746522815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58101680 0x7fee58101b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009990 tx=0x7fee4c009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.835+0000 7fee5d67b700 1 -- 172.21.15.27:0/1746522815 shutdown_connections 2023-12-17T12:09:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.835+0000 7fee5d67b700 1 --2- 172.21.15.27:0/1746522815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee58102040 0x7fee5810e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.835+0000 7fee5d67b700 1 --2- 172.21.15.27:0/1746522815 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee581076f0 0x7fee58107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.835+0000 7fee5d67b700 1 --2- 172.21.15.27:0/1746522815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58101680 0x7fee58101b00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.835+0000 7fee5d67b700 1 -- 172.21.15.27:0/1746522815 >> 172.21.15.27:0/1746522815 conn(0x7fee580787c0 msgr2=0x7fee58078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.835+0000 7fee5d67b700 1 -- 172.21.15.27:0/1746522815 shutdown_connections 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.836+0000 7fee5d67b700 1 -- 172.21.15.27:0/1746522815 wait complete. 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.837+0000 7fee5d67b700 1 Processor -- start 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.837+0000 7fee5d67b700 1 -- start start 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.837+0000 7fee5d67b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58101680 0x7fee5819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.837+0000 7fee5d67b700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee58102040 0x7fee5819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.837+0000 7fee5d67b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee581076f0 0x7fee58199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.837+0000 7fee5d67b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5819a2b0 con 0x7fee58101680 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.837+0000 7fee5d67b700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5819a420 con 0x7fee58102040 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.837+0000 7fee5d67b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5819a5e0 con 0x7fee581076f0 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee567fc700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee58102040 0x7fee5819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee567fc700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee58102040 0x7fee5819ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:56552/0 (socket says 172.21.15.27:56552) 2023-12-17T12:09:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee567fc700 1 -- 172.21.15.27:0/1654801303 learned_addr learned my addr 172.21.15.27:0/1654801303 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:20.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee577fe700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee581076f0 0x7fee58199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:20.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee56ffd700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58101680 0x7fee5819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:20.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee567fc700 1 -- 172.21.15.27:0/1654801303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee581076f0 msgr2=0x7fee58199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:20.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee567fc700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee581076f0 0x7fee58199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:20.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee567fc700 1 -- 172.21.15.27:0/1654801303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58101680 msgr2=0x7fee5819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:20.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee567fc700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58101680 0x7fee5819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:20.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.838+0000 7fee567fc700 1 -- 172.21.15.27:0/1654801303 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0095f0 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.839+0000 7fee567fc700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee58102040 0x7fee5819ec80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee4c005770 tx=0x7fee4c0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.839+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c01d070 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.839+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5819a860 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.839+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee4c022470 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.839+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c017650 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.840+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581a67d0 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.841+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee58074320 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.843+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fee4c00fb40 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.845+0000 7fee3ffff700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee400a1510 0x7fee400a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.845+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fee4c0ce280 con 0x7fee58102040 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.845+0000 7fee56ffd700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee400a1510 0x7fee400a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.845+0000 7fee3ffff700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] conn(0x7fee400aacf0 0x7fee400ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.845+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 --> [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee400ad7c0 con 0x7fee400aacf0 2023-12-17T12:09:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.845+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee4c0ce5f0 con 0x7fee58102040 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.846+0000 7fee577fe700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] conn(0x7fee400aacf0 0x7fee400ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.846+0000 7fee56ffd700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee400a1510 0x7fee400a39d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee48009c60 tx=0x7fee48009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.846+0000 7fee577fe700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] conn(0x7fee400aacf0 0x7fee400ad110 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.848+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 <== osd.7 v2:172.21.15.141:6816/303714456 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fee400ad7c0 con 0x7fee400aacf0 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.866+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 --> [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee58106400 con 0x7fee400aacf0 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.867+0000 7fee3ffff700 1 -- 172.21.15.27:0/1654801303 <== osd.7 v2:172.21.15.141:6816/303714456 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fee58106400 con 0x7fee400aacf0 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 >> [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] conn(0x7fee400aacf0 msgr2=0x7fee400ad110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] conn(0x7fee400aacf0 0x7fee400ad110 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee400a1510 msgr2=0x7fee400a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee400a1510 0x7fee400a39d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee48009c60 tx=0x7fee48009260 comp rx=0 tx=0).stop 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee58102040 msgr2=0x7fee5819ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 --2- 172.21.15.27:0/1654801303 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fee58102040 0x7fee5819ec80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee4c005770 tx=0x7fee4c0057a0 comp rx=0 tx=0).stop 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee577fe700 1 -- 172.21.15.27:0/1654801303 reap_dead start 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 shutdown_connections 2023-12-17T12:09:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 >> 172.21.15.27:0/1654801303 conn(0x7fee580787c0 msgr2=0x7fee580ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:20.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 shutdown_connections 2023-12-17T12:09:20.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:19.868+0000 7fee5d67b700 1 -- 172.21.15.27:0/1654801303 wait complete. 2023-12-17T12:09:20.567 INFO:teuthology.orchestra.run.smithi027.stdout:34359738435 2023-12-17T12:09:20.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-17T12:09:20.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:20 smithi027 ceph-mon[36871]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-12-17T12:09:20.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:20 smithi141 ceph-mon[41225]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-12-17T12:09:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:20.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:20 smithi192 ceph-mon[41899]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-12-17T12:09:20.817 INFO:teuthology.orchestra.run.smithi027.stdout:240518168607 2023-12-17T12:09:20.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-12-17T12:09:21.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.027+0000 7f0bde090700 1 -- 172.21.15.27:0/1513553191 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd810e0e0 msgr2=0x7f0bd81104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.027+0000 7f0bde090700 1 --2- 172.21.15.27:0/1513553191 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd810e0e0 0x7f0bd81104d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009f90 tx=0x7f0bd000b330 comp rx=0 tx=0).stop 2023-12-17T12:09:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.027+0000 7f0bde090700 1 -- 172.21.15.27:0/1513553191 shutdown_connections 2023-12-17T12:09:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.027+0000 7f0bde090700 1 --2- 172.21.15.27:0/1513553191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd8068f70 0x7f0bd8069350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.027+0000 7f0bde090700 1 --2- 172.21.15.27:0/1513553191 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd810e0e0 0x7f0bd81104d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.027+0000 7f0bde090700 1 --2- 172.21.15.27:0/1513553191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd8069890 0x7f0bd810dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.027+0000 7f0bde090700 1 -- 172.21.15.27:0/1513553191 >> 172.21.15.27:0/1513553191 conn(0x7f0bd8076570 msgr2=0x7f0bd8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.028+0000 7f0bde090700 1 -- 172.21.15.27:0/1513553191 shutdown_connections 2023-12-17T12:09:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.028+0000 7f0bde090700 1 -- 172.21.15.27:0/1513553191 wait complete. 2023-12-17T12:09:21.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.030+0000 7f0bde090700 1 Processor -- start 2023-12-17T12:09:21.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.030+0000 7f0bde090700 1 -- start start 2023-12-17T12:09:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.030+0000 7f0bde090700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd8068f70 0x7f0bd819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bde090700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd8069890 0x7f0bd819fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bde090700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd810e0e0 0x7f0bd8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bde090700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd819a120 con 0x7f0bd810e0e0 2023-12-17T12:09:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bde090700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd819a290 con 0x7f0bd8068f70 2023-12-17T12:09:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bde090700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd819a450 con 0x7f0bd8069890 2023-12-17T12:09:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bd77fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd8068f70 0x7f0bd819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd8069890 0x7f0bd819fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bd77fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd8068f70 0x7f0bd819f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:56570/0 (socket says 172.21.15.27:56570) 2023-12-17T12:09:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bd77fe700 1 -- 172.21.15.27:0/4222814331 learned_addr learned my addr 172.21.15.27:0/4222814331 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.031+0000 7f0bd7fff700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd810e0e0 0x7f0bd8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.032+0000 7f0bd6ffd700 1 -- 172.21.15.27:0/4222814331 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd8068f70 msgr2=0x7f0bd819f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.032+0000 7f0bd6ffd700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd8068f70 0x7f0bd819f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.032+0000 7f0bd6ffd700 1 -- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd810e0e0 msgr2=0x7f0bd8199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.032+0000 7f0bd6ffd700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd810e0e0 0x7f0bd8199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.032+0000 7f0bd6ffd700 1 -- 172.21.15.27:0/4222814331 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40095f0 con 0x7f0bd8069890 2023-12-17T12:09:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.032+0000 7f0bd7fff700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd810e0e0 0x7f0bd8199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.032+0000 7f0bd77fe700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0bd8068f70 0x7f0bd819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.032+0000 7f0bd6ffd700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd8069890 0x7f0bd819fec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400eb90 tx=0x7f0bc400ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.033+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400cda0 con 0x7f0bd8069890 2023-12-17T12:09:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.033+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bc40044b0 con 0x7f0bd8069890 2023-12-17T12:09:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.033+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4010430 con 0x7f0bd8069890 2023-12-17T12:09:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.033+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd0009c40 con 0x7f0bd8069890 2023-12-17T12:09:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.033+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd81a67a0 con 0x7f0bd8069890 2023-12-17T12:09:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.034+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0bd8113860 con 0x7f0bd8069890 2023-12-17T12:09:21.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.037+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f0bc4003f20 con 0x7f0bd8069890 2023-12-17T12:09:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.039+0000 7f0bd4ff9700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bbc0a1510 0x7f0bbc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.039+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0bc4014070 con 0x7f0bd8069890 2023-12-17T12:09:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.039+0000 7f0bd77fe700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bbc0a1510 0x7f0bbc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.040+0000 7f0bd4ff9700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] conn(0x7f0bbc0aacf0 0x7f0bbc0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.040+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 --> [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bbc0ad7c0 con 0x7f0bbc0aacf0 2023-12-17T12:09:21.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.040+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc40cd5e0 con 0x7f0bd8069890 2023-12-17T12:09:21.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.040+0000 7f0bd7fff700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] conn(0x7f0bbc0aacf0 0x7f0bbc0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.040+0000 7f0bd77fe700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bbc0a1510 0x7f0bbc0a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009ca0 tx=0x7f0bc8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:21.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.041+0000 7f0bd7fff700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] conn(0x7f0bbc0aacf0 0x7f0bbc0ad110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:21.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.042+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 <== osd.1 v2:172.21.15.141:6800/2130638652 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0bbc0ad7c0 con 0x7f0bbc0aacf0 2023-12-17T12:09:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.060+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 --> [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0bd8061cd0 con 0x7f0bbc0aacf0 2023-12-17T12:09:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.061+0000 7f0bd4ff9700 1 -- 172.21.15.27:0/4222814331 <== osd.1 v2:172.21.15.141:6800/2130638652 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0bd8061cd0 con 0x7f0bbc0aacf0 2023-12-17T12:09:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 >> [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] conn(0x7f0bbc0aacf0 msgr2=0x7f0bbc0ad110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] conn(0x7f0bbc0aacf0 0x7f0bbc0ad110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bbc0a1510 msgr2=0x7f0bbc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0bbc0a1510 0x7f0bbc0a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009ca0 tx=0x7f0bc8009260 comp rx=0 tx=0).stop 2023-12-17T12:09:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd8069890 msgr2=0x7f0bd819fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 --2- 172.21.15.27:0/4222814331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd8069890 0x7f0bd819fec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400eb90 tx=0x7f0bc400ef50 comp rx=0 tx=0).stop 2023-12-17T12:09:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bd7fff700 1 -- 172.21.15.27:0/4222814331 reap_dead start 2023-12-17T12:09:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 shutdown_connections 2023-12-17T12:09:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 >> 172.21.15.27:0/4222814331 conn(0x7f0bd8076570 msgr2=0x7f0bd810f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 shutdown_connections 2023-12-17T12:09:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.062+0000 7f0bde090700 1 -- 172.21.15.27:0/4222814331 wait complete. 2023-12-17T12:09:21.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 -- 172.21.15.27:0/1346623814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 msgr2=0x7f16b8102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 --2- 172.21.15.27:0/1346623814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 0x7f16b8102960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f16b40099c0 tx=0x7f16b4009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 -- 172.21.15.27:0/1346623814 shutdown_connections 2023-12-17T12:09:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 --2- 172.21.15.27:0/1346623814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16b8107b10 0x7f16b810e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 --2- 172.21.15.27:0/1346623814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 0x7f16b8102960 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 --2- 172.21.15.27:0/1346623814 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f16b8102f30 0x7f16b81073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 -- 172.21.15.27:0/1346623814 >> 172.21.15.27:0/1346623814 conn(0x7f16b80fbe10 msgr2=0x7f16b80fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 -- 172.21.15.27:0/1346623814 shutdown_connections 2023-12-17T12:09:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.262+0000 7f16bf368700 1 -- 172.21.15.27:0/1346623814 wait complete. 2023-12-17T12:09:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.263+0000 7f16bf368700 1 Processor -- start 2023-12-17T12:09:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.263+0000 7f16bf368700 1 -- start start 2023-12-17T12:09:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.264+0000 7f16bf368700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 0x7f16b819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.264+0000 7f16bf368700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16b8102f30 0x7f16b819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.264+0000 7f16bf368700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f16b8107b10 0x7f16b8199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.264+0000 7f16bf368700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8199eb0 con 0x7f16b8102580 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.264+0000 7f16bf368700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b819a020 con 0x7f16b8107b10 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.264+0000 7f16bf368700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b819a1e0 con 0x7f16b8102f30 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16be366700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 0x7f16b819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16be366700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 0x7f16b819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35590/0 (socket says 172.21.15.27:35590) 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16be366700 1 -- 172.21.15.27:0/685782939 learned_addr learned my addr 172.21.15.27:0/685782939 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16beb67700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f16b8107b10 0x7f16b8199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16bdb65700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16b8102f30 0x7f16b819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16be366700 1 -- 172.21.15.27:0/685782939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16b8102f30 msgr2=0x7f16b819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16be366700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16b8102f30 0x7f16b819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16be366700 1 -- 172.21.15.27:0/685782939 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f16b8107b10 msgr2=0x7f16b8199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16be366700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f16b8107b10 0x7f16b8199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16be366700 1 -- 172.21.15.27:0/685782939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b40095f0 con 0x7f16b8102580 2023-12-17T12:09:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.265+0000 7f16beb67700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f16b8107b10 0x7f16b8199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.266+0000 7f16be366700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 0x7f16b819f700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16b40095c0 tx=0x7f16b40057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.266+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b401d070 con 0x7f16b8102580 2023-12-17T12:09:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.266+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b819a460 con 0x7f16b8102580 2023-12-17T12:09:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.266+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16b400fc80 con 0x7f16b8102580 2023-12-17T12:09:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.266+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4022cf0 con 0x7f16b8102580 2023-12-17T12:09:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.267+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b81a66f0 con 0x7f16b8102580 2023-12-17T12:09:21.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.268+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16b81115c0 con 0x7f16b8102580 2023-12-17T12:09:21.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.271+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f16b4022470 con 0x7f16b8102580 2023-12-17T12:09:21.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.273+0000 7f16af7fe700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16a40a1610 0x7f16a40a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.273+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f16b40cdf10 con 0x7f16b8102580 2023-12-17T12:09:21.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.273+0000 7f16bdb65700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16a40a1610 0x7f16a40a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.273+0000 7f16af7fe700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] conn(0x7f16a40aacc0 0x7f16a40ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:21.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.273+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 --> [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16a40ad790 con 0x7f16a40aacc0 2023-12-17T12:09:21.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.273+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b40d0530 con 0x7f16b8102580 2023-12-17T12:09:21.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.274+0000 7f16beb67700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] conn(0x7f16a40aacc0 0x7f16a40ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:21.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.274+0000 7f16bdb65700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16a40a1610 0x7f16a40a3ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16a8005fd0 tx=0x7f16a8005f60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:21.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.274+0000 7f16beb67700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] conn(0x7f16a40aacc0 0x7f16a40ad0e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:21.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.276+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 <== osd.9 v2:172.21.15.27:6826/3328001767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f16a40ad790 con 0x7f16a40aacc0 2023-12-17T12:09:21.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.308+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 --> [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f16b8061cd0 con 0x7f16a40aacc0 2023-12-17T12:09:21.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.309+0000 7f16af7fe700 1 -- 172.21.15.27:0/685782939 <== osd.9 v2:172.21.15.27:6826/3328001767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f16b8061cd0 con 0x7f16a40aacc0 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.309+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] conn(0x7f16a40aacc0 msgr2=0x7f16a40ad0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.309+0000 7f16bf368700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] conn(0x7f16a40aacc0 0x7f16a40ad0e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16a40a1610 msgr2=0x7f16a40a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16bf368700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f16a40a1610 0x7f16a40a3ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16a8005fd0 tx=0x7f16a8005f60 comp rx=0 tx=0).stop 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 msgr2=0x7f16b819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16bf368700 1 --2- 172.21.15.27:0/685782939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16b8102580 0x7f16b819f700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16b40095c0 tx=0x7f16b40057d0 comp rx=0 tx=0).stop 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16beb67700 1 -- 172.21.15.27:0/685782939 reap_dead start 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 shutdown_connections 2023-12-17T12:09:21.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 >> 172.21.15.27:0/685782939 conn(0x7f16b80fbe10 msgr2=0x7f16b8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:21.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 shutdown_connections 2023-12-17T12:09:21.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:21.310+0000 7f16bf368700 1 -- 172.21.15.27:0/685782939 wait complete. 2023-12-17T12:09:22.193 INFO:teuthology.orchestra.run.smithi027.stdout:296352743446 2023-12-17T12:09:22.193 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-12-17T12:09:22.560 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:22.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:22 smithi141 ceph-mon[41225]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 7 op/s 2023-12-17T12:09:22.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:22 smithi192 ceph-mon[41899]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 7 op/s 2023-12-17T12:09:22.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:22 smithi027 ceph-mon[36871]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 7 op/s 2023-12-17T12:09:22.823 INFO:teuthology.orchestra.run.smithi027.stdout:210453397540 2023-12-17T12:09:22.823 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-17T12:09:22.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.844+0000 7f5f0b817700 1 -- 172.21.15.27:0/3827868837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 msgr2=0x7f5f04101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:22.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.844+0000 7f5f0b817700 1 --2- 172.21.15.27:0/3827868837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 0x7f5f04101b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009990 tx=0x7f5ef4009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.845+0000 7f5f0b817700 1 -- 172.21.15.27:0/3827868837 shutdown_connections 2023-12-17T12:09:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.845+0000 7f5f0b817700 1 --2- 172.21.15.27:0/3827868837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f04102040 0x7f5f0410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.845+0000 7f5f0b817700 1 --2- 172.21.15.27:0/3827868837 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f5f041076f0 0x7f5f04107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.845+0000 7f5f0b817700 1 --2- 172.21.15.27:0/3827868837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 0x7f5f04101b00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.845+0000 7f5f0b817700 1 -- 172.21.15.27:0/3827868837 >> 172.21.15.27:0/3827868837 conn(0x7f5f040787c0 msgr2=0x7f5f04078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.845+0000 7f5f0b817700 1 -- 172.21.15.27:0/3827868837 shutdown_connections 2023-12-17T12:09:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.845+0000 7f5f0b817700 1 -- 172.21.15.27:0/3827868837 wait complete. 2023-12-17T12:09:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.846+0000 7f5f0b817700 1 Processor -- start 2023-12-17T12:09:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.846+0000 7f5f0b817700 1 -- start start 2023-12-17T12:09:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f0b817700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 0x7f5f0419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f0b817700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f5f04102040 0x7f5f0419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f0b817700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f041076f0 0x7f5f041999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f0b817700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0419a010 con 0x7f5f04101680 2023-12-17T12:09:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f0b817700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0419a180 con 0x7f5f04102040 2023-12-17T12:09:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f0b817700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0419a340 con 0x7f5f041076f0 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f095b3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 0x7f5f0419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f095b3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 0x7f5f0419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35608/0 (socket says 172.21.15.27:35608) 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f095b3700 1 -- 172.21.15.27:0/2019380424 learned_addr learned my addr 172.21.15.27:0/2019380424 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f09db4700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f041076f0 0x7f5f041999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f08db2700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f5f04102040 0x7f5f0419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f095b3700 1 -- 172.21.15.27:0/2019380424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f041076f0 msgr2=0x7f5f041999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f095b3700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f041076f0 0x7f5f041999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f095b3700 1 -- 172.21.15.27:0/2019380424 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f5f04102040 msgr2=0x7f5f0419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f095b3700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f5f04102040 0x7f5f0419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f095b3700 1 -- 172.21.15.27:0/2019380424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40095f0 con 0x7f5f04101680 2023-12-17T12:09:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.847+0000 7f5f08db2700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f5f04102040 0x7f5f0419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:22.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.848+0000 7f5f095b3700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 0x7f5f0419f870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000e9a0 tx=0x7f5f0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:22.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.848+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00009790 con 0x7f5f04101680 2023-12-17T12:09:22.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.848+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f0000cbd0 con 0x7f5f04101680 2023-12-17T12:09:22.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.848+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0419a620 con 0x7f5f04101680 2023-12-17T12:09:22.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.848+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00018680 con 0x7f5f04101680 2023-12-17T12:09:22.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.848+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f041a6860 con 0x7f5f04101680 2023-12-17T12:09:22.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.849+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f04111650 con 0x7f5f04101680 2023-12-17T12:09:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.852+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f5f0000cd40 con 0x7f5f04101680 2023-12-17T12:09:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.853+0000 7f5efa7fc700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5ef00a1510 0x7f5ef00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.853+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5f00014070 con 0x7f5f04101680 2023-12-17T12:09:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.853+0000 7f5f08db2700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5ef00a1510 0x7f5ef00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.854+0000 7f5efa7fc700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] conn(0x7f5ef00aacf0 0x7f5ef00ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.854+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 --> [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ef00ad7c0 con 0x7f5ef00aacf0 2023-12-17T12:09:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.854+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f000cd7a0 con 0x7f5f04101680 2023-12-17T12:09:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.854+0000 7f5f09db4700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] conn(0x7f5ef00aacf0 0x7f5ef00ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.854+0000 7f5f08db2700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5ef00a1510 0x7f5ef00a39d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400b2e0 tx=0x7f5ef4005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.855+0000 7f5f09db4700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] conn(0x7f5ef00aacf0 0x7f5ef00ad110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.856+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 <== osd.6 v2:172.21.15.27:6818/3435247687 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5ef00ad7c0 con 0x7f5ef00aacf0 2023-12-17T12:09:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.873+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 --> [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f04061cd0 con 0x7f5ef00aacf0 2023-12-17T12:09:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.874+0000 7f5efa7fc700 1 -- 172.21.15.27:0/2019380424 <== osd.6 v2:172.21.15.27:6818/3435247687 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5f04061cd0 con 0x7f5ef00aacf0 2023-12-17T12:09:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.874+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] conn(0x7f5ef00aacf0 msgr2=0x7f5ef00ad110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.874+0000 7f5f0b817700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] conn(0x7f5ef00aacf0 0x7f5ef00ad110 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5ef00a1510 msgr2=0x7f5ef00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f0b817700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f5ef00a1510 0x7f5ef00a39d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400b2e0 tx=0x7f5ef4005fd0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 msgr2=0x7f5f0419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f0b817700 1 --2- 172.21.15.27:0/2019380424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f04101680 0x7f5f0419f870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000e9a0 tx=0x7f5f0000ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f09db4700 1 -- 172.21.15.27:0/2019380424 reap_dead start 2023-12-17T12:09:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 shutdown_connections 2023-12-17T12:09:22.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 >> 172.21.15.27:0/2019380424 conn(0x7f5f040787c0 msgr2=0x7f5f0410cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:22.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 shutdown_connections 2023-12-17T12:09:22.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:22.875+0000 7f5f0b817700 1 -- 172.21.15.27:0/2019380424 wait complete. 2023-12-17T12:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout:188978561066 2023-12-17T12:09:23.338 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-17T12:09:23.526 INFO:teuthology.orchestra.run.smithi027.stdout:274877906970 2023-12-17T12:09:23.527 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-12-17T12:09:23.558 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:23.604 INFO:teuthology.orchestra.run.smithi027.stdout:55834574911 2023-12-17T12:09:23.605 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-17T12:09:23.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-17T12:09:23.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-12-17T12:09:23.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-12-17T12:09:23.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2023-12-17T12:09:23.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-12-17T12:09:23.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-17T12:09:23.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-17T12:09:23.717 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:23 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-12-17T12:09:23.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-17T12:09:23.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-12-17T12:09:23.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-12-17T12:09:23.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2023-12-17T12:09:23.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-12-17T12:09:23.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-17T12:09:23.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-17T12:09:23.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:23 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-12-17T12:09:23.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-17T12:09:23.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-12-17T12:09:23.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-12-17T12:09:23.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2023-12-17T12:09:23.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-12-17T12:09:23.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-17T12:09:23.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-17T12:09:23.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:23 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-12-17T12:09:23.947 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:24.259 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:24.313 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:24.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-12-17T12:09:24.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:24 smithi192 ceph-mon[41899]: osdmap e83: 12 total, 12 up, 12 in 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-12-17T12:09:24.826 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:24 smithi027 ceph-mon[36871]: osdmap e83: 12 total, 12 up, 12 in 2023-12-17T12:09:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-12-17T12:09:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-17T12:09:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-12-17T12:09:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-12-17T12:09:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2023-12-17T12:09:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-12-17T12:09:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-17T12:09:25.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-17T12:09:25.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-12-17T12:09:25.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:24 smithi141 ceph-mon[41225]: osdmap e83: 12 total, 12 up, 12 in 2023-12-17T12:09:25.021 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:25.723 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:25 smithi027 ceph-mon[36871]: osdmap e84: 12 total, 12 up, 12 in 2023-12-17T12:09:25.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:25 smithi192 ceph-mon[41899]: osdmap e84: 12 total, 12 up, 12 in 2023-12-17T12:09:25.909 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:26.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:25 smithi141 ceph-mon[41225]: osdmap e84: 12 total, 12 up, 12 in 2023-12-17T12:09:26.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:26 smithi192 ceph-mon[41899]: pgmap v285: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 2/192 objects degraded (1.042%) 2023-12-17T12:09:26.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:26 smithi192 ceph-mon[41899]: Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED) 2023-12-17T12:09:26.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:26 smithi027 ceph-mon[36871]: pgmap v285: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 2/192 objects degraded (1.042%) 2023-12-17T12:09:26.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:26 smithi027 ceph-mon[36871]: Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED) 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.931+0000 7f0e8f998700 1 -- 172.21.15.27:0/3133306660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88069930 msgr2=0x7f0e8810e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.931+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3133306660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88069930 0x7f0e8810e410 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b0d0 tx=0x7f0e8000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.931+0000 7f0e8f998700 1 -- 172.21.15.27:0/3133306660 shutdown_connections 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.931+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3133306660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e88107640 0x7f0e88107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.931+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3133306660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88069930 0x7f0e8810e410 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.931+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3133306660 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0e88068f70 0x7f0e880693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.931+0000 7f0e8f998700 1 -- 172.21.15.27:0/3133306660 >> 172.21.15.27:0/3133306660 conn(0x7f0e88076570 msgr2=0x7f0e88076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.931+0000 7f0e8f998700 1 -- 172.21.15.27:0/3133306660 shutdown_connections 2023-12-17T12:09:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.932+0000 7f0e8f998700 1 -- 172.21.15.27:0/3133306660 wait complete. 2023-12-17T12:09:26.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.933+0000 7f0e8f998700 1 Processor -- start 2023-12-17T12:09:26.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.933+0000 7f0e8f998700 1 -- start start 2023-12-17T12:09:26.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8f998700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0e88068f70 0x7f0e8819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:26.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8f998700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e88069930 0x7f0e8819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:26.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8f998700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88107640 0x7f0e88197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:26.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8f998700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88197e90 con 0x7f0e88107640 2023-12-17T12:09:26.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8f998700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88198000 con 0x7f0e88068f70 2023-12-17T12:09:26.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8f998700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e881981c0 con 0x7f0e88069930 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8d734700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0e88068f70 0x7f0e8819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8d734700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0e88068f70 0x7f0e8819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:59074/0 (socket says 172.21.15.27:59074) 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.934+0000 7f0e8d734700 1 -- 172.21.15.27:0/3105564944 learned_addr learned my addr 172.21.15.27:0/3105564944 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8df35700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88107640 0x7f0e88197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8cf33700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e88069930 0x7f0e8819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8df35700 1 -- 172.21.15.27:0/3105564944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e88069930 msgr2=0x7f0e8819dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8df35700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e88069930 0x7f0e8819dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8df35700 1 -- 172.21.15.27:0/3105564944 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0e88068f70 msgr2=0x7f0e8819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8df35700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0e88068f70 0x7f0e8819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8df35700 1 -- 172.21.15.27:0/3105564944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80009d00 con 0x7f0e88107640 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8d734700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0e88068f70 0x7f0e8819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e8df35700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88107640 0x7f0e88197820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b770 tx=0x7f0e8000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.935+0000 7f0e7e7fc700 1 -- 172.21.15.27:0/3105564944 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000e040 con 0x7f0e88107640 2023-12-17T12:09:26.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.936+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e88198440 con 0x7f0e88107640 2023-12-17T12:09:26.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.936+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e881a46d0 con 0x7f0e88107640 2023-12-17T12:09:26.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.936+0000 7f0e7e7fc700 1 -- 172.21.15.27:0/3105564944 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e80007e00 con 0x7f0e88107640 2023-12-17T12:09:26.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.936+0000 7f0e7e7fc700 1 -- 172.21.15.27:0/3105564944 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8001dd50 con 0x7f0e88107640 2023-12-17T12:09:26.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.937+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e88109af0 con 0x7f0e88107640 2023-12-17T12:09:26.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.939+0000 7f0e7e7fc700 1 -- 172.21.15.27:0/3105564944 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f0e80004c60 con 0x7f0e88107640 2023-12-17T12:09:26.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.940+0000 7f0e7e7fc700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0e740a1510 0x7f0e740a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:26.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.940+0000 7f0e7e7fc700 1 -- 172.21.15.27:0/3105564944 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0e800ce240 con 0x7f0e88107640 2023-12-17T12:09:26.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.940+0000 7f0e8d734700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0e740a1510 0x7f0e740a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:26.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.941+0000 7f0e8d734700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0e740a1510 0x7f0e740a39d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0e84011550 tx=0x7f0e84009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:26.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:26.945+0000 7f0e7e7fc700 1 -- 172.21.15.27:0/3105564944 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0e80097780 con 0x7f0e88107640 2023-12-17T12:09:27.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:26 smithi141 ceph-mon[41225]: pgmap v285: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 2/192 objects degraded (1.042%) 2023-12-17T12:09:27.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:26 smithi141 ceph-mon[41225]: Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED) 2023-12-17T12:09:27.039 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:27.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.209+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f0e8804f290 con 0x7f0e88107640 2023-12-17T12:09:27.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.210+0000 7f0e7e7fc700 1 -- 172.21.15.27:0/3105564944 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f0e8009ada0 con 0x7f0e88107640 2023-12-17T12:09:27.211 INFO:teuthology.orchestra.run.smithi027.stdout:330712481811 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0e740a1510 msgr2=0x7f0e740a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0e740a1510 0x7f0e740a39d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0e84011550 tx=0x7f0e84009260 comp rx=0 tx=0).stop 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88107640 msgr2=0x7f0e88197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88107640 0x7f0e88197820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000b770 tx=0x7f0e8000cce0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 shutdown_connections 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f0e740a1510 0x7f0e740a39d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e88107640 0x7f0e88197820 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f0e88068f70 0x7f0e8819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 --2- 172.21.15.27:0/3105564944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e88069930 0x7f0e8819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 >> 172.21.15.27:0/3105564944 conn(0x7f0e88076570 msgr2=0x7f0e88105590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.212+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 shutdown_connections 2023-12-17T12:09:27.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.213+0000 7f0e8f998700 1 -- 172.21.15.27:0/3105564944 wait complete. 2023-12-17T12:09:27.220 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.644+0000 7f7169526700 1 -- 172.21.15.27:0/3193185045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7164101680 msgr2=0x7f7164101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.644+0000 7f7169526700 1 --2- 172.21.15.27:0/3193185045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7164101680 0x7f7164101b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7158009990 tx=0x7f7158009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.644+0000 7f7169526700 1 -- 172.21.15.27:0/3193185045 shutdown_connections 2023-12-17T12:09:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.644+0000 7f7169526700 1 --2- 172.21.15.27:0/3193185045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71641076f0 0x7f7164107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.644+0000 7f7169526700 1 --2- 172.21.15.27:0/3193185045 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7164102040 0x7f716410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.644+0000 7f7169526700 1 --2- 172.21.15.27:0/3193185045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7164101680 0x7f7164101b00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.644+0000 7f7169526700 1 -- 172.21.15.27:0/3193185045 >> 172.21.15.27:0/3193185045 conn(0x7f71640787c0 msgr2=0x7f7164078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:27.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.645+0000 7f7169526700 1 -- 172.21.15.27:0/3193185045 shutdown_connections 2023-12-17T12:09:27.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.645+0000 7f7169526700 1 -- 172.21.15.27:0/3193185045 wait complete. 2023-12-17T12:09:27.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.646+0000 7f7169526700 1 Processor -- start 2023-12-17T12:09:27.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.646+0000 7f7169526700 1 -- start start 2023-12-17T12:09:27.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.647+0000 7f7169526700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7164101680 0x7f716419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:27.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.647+0000 7f7169526700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164102040 0x7f716419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:27.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.647+0000 7f7169526700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71641076f0 0x7f7164197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:27.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.647+0000 7f7169526700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164197df0 con 0x7f71641076f0 2023-12-17T12:09:27.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.647+0000 7f7169526700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164197f60 con 0x7f7164101680 2023-12-17T12:09:27.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.647+0000 7f7169526700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164198120 con 0x7f7164102040 2023-12-17T12:09:27.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.648+0000 7f71637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71641076f0 0x7f7164197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.648+0000 7f71637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71641076f0 0x7f7164197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58522/0 (socket says 172.21.15.27:58522) 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.648+0000 7f71637fe700 1 -- 172.21.15.27:0/3052662708 learned_addr learned my addr 172.21.15.27:0/3052662708 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.648+0000 7f7162ffd700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7164101680 0x7f716419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.648+0000 7f71627fc700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164102040 0x7f716419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.648+0000 7f71637fe700 1 -- 172.21.15.27:0/3052662708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164102040 msgr2=0x7f716419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f71637fe700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164102040 0x7f716419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f71637fe700 1 -- 172.21.15.27:0/3052662708 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7164101680 msgr2=0x7f716419d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f71637fe700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7164101680 0x7f716419d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f71637fe700 1 -- 172.21.15.27:0/3052662708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71580095f0 con 0x7f71641076f0 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f71637fe700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71641076f0 0x7f7164197790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f715c007f10 tx=0x7f715c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f714bfff700 1 -- 172.21.15.27:0/3052662708 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c017070 con 0x7f71641076f0 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f714bfff700 1 -- 172.21.15.27:0/3052662708 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f715c00f040 con 0x7f71641076f0 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7164198400 con 0x7f71641076f0 2023-12-17T12:09:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.649+0000 7f714bfff700 1 -- 172.21.15.27:0/3052662708 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c005570 con 0x7f71641076f0 2023-12-17T12:09:27.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.650+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71641a45f0 con 0x7f71641076f0 2023-12-17T12:09:27.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.651+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7164109b80 con 0x7f71641076f0 2023-12-17T12:09:27.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.653+0000 7f714bfff700 1 -- 172.21.15.27:0/3052662708 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f715c00db20 con 0x7f71641076f0 2023-12-17T12:09:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.655+0000 7f714bfff700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f714c0a1590 0x7f714c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.655+0000 7f714bfff700 1 -- 172.21.15.27:0/3052662708 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f715c0cc530 con 0x7f71641076f0 2023-12-17T12:09:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.655+0000 7f7162ffd700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f714c0a1590 0x7f714c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.656+0000 7f7162ffd700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f714c0a1590 0x7f714c0a3a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7154005950 tx=0x7f715400b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:27.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.661+0000 7f714bfff700 1 -- 172.21.15.27:0/3052662708 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f715c0959e0 con 0x7f71641076f0 2023-12-17T12:09:27.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:27 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3105564944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-17T12:09:27.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:27 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3105564944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-17T12:09:27.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.880+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7164102d10 con 0x7f71641076f0 2023-12-17T12:09:27.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.881+0000 7f714bfff700 1 -- 172.21.15.27:0/3052662708 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f715c099000 con 0x7f71641076f0 2023-12-17T12:09:27.882 INFO:teuthology.orchestra.run.smithi027.stdout:107374182455 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.883+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f714c0a1590 msgr2=0x7f714c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.883+0000 7f7169526700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f714c0a1590 0x7f714c0a3a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7154005950 tx=0x7f715400b410 comp rx=0 tx=0).stop 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.883+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71641076f0 msgr2=0x7f7164197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.883+0000 7f7169526700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71641076f0 0x7f7164197790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f715c007f10 tx=0x7f715c00d3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.884+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 shutdown_connections 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.884+0000 7f7169526700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f714c0a1590 0x7f714c0a3a50 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7154005950 tx=0x7f715400b410 comp rx=0 tx=0).stop 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.884+0000 7f7169526700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71641076f0 0x7f7164197790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.884+0000 7f7169526700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7164101680 0x7f716419d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.884+0000 7f7169526700 1 --2- 172.21.15.27:0/3052662708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164102040 0x7f716419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:27.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.884+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 >> 172.21.15.27:0/3052662708 conn(0x7f71640787c0 msgr2=0x7f71641003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:27.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.884+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 shutdown_connections 2023-12-17T12:09:27.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:27.884+0000 7f7169526700 1 -- 172.21.15.27:0/3052662708 wait complete. 2023-12-17T12:09:28.013 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:27 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3105564944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-17T12:09:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:28 smithi027 ceph-mon[36871]: pgmap v286: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 2/192 objects degraded (1.042%) 2023-12-17T12:09:28.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:28 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3052662708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-17T12:09:28.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:28 smithi192 ceph-mon[41899]: pgmap v286: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 2/192 objects degraded (1.042%) 2023-12-17T12:09:28.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:28 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3052662708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-17T12:09:29.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:28 smithi141 ceph-mon[41225]: pgmap v286: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 2/192 objects degraded (1.042%) 2023-12-17T12:09:29.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:28 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3052662708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-17T12:09:29.076 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481809 got 330712481811 for osd.11 2023-12-17T12:09:29.077 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:29.706 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182453 got 107374182455 for osd.3 2023-12-17T12:09:29.707 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:30.621 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:30 smithi027 ceph-mon[36871]: pgmap v287: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3 op/s; 2/192 objects degraded (1.042%); 88 B/s, 0 keys/s, 0 objects/s recovering 2023-12-17T12:09:30.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.647+0000 7ff55bfff700 1 -- 172.21.15.27:0/3431130663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554003970 con 0x7ff564068f70 2023-12-17T12:09:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.648+0000 7ff56b3f6700 1 -- 172.21.15.27:0/3431130663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff564068f70 msgr2=0x7ff5640693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.648+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/3431130663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff564068f70 0x7ff5640693f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5540099c0 tx=0x7ff554009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.648+0000 7ff56b3f6700 1 -- 172.21.15.27:0/3431130663 shutdown_connections 2023-12-17T12:09:30.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.648+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/3431130663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff564069930 0x7ff56410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.648+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/3431130663 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff564107620 0x7ff564107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.648+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/3431130663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff564068f70 0x7ff5640693f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.648+0000 7ff56b3f6700 1 -- 172.21.15.27:0/3431130663 >> 172.21.15.27:0/3431130663 conn(0x7ff564076570 msgr2=0x7ff564076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:30.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.648+0000 7ff56b3f6700 1 -- 172.21.15.27:0/3431130663 shutdown_connections 2023-12-17T12:09:30.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.649+0000 7ff56b3f6700 1 -- 172.21.15.27:0/3431130663 wait complete. 2023-12-17T12:09:30.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.650+0000 7ff56b3f6700 1 Processor -- start 2023-12-17T12:09:30.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.651+0000 7ff56b3f6700 1 -- start start 2023-12-17T12:09:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.652+0000 7ff56b3f6700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff564069930 0x7ff56406b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.652+0000 7ff56b3f6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff564107620 0x7ff56406b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.653+0000 7ff569192700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff564069930 0x7ff56406b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.653+0000 7ff569192700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff564069930 0x7ff56406b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:59156/0 (socket says 172.21.15.27:59156) 2023-12-17T12:09:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.653+0000 7ff56b3f6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56406e660 0x7ff56406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.653+0000 7ff568991700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff564107620 0x7ff56406b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.653+0000 7ff568991700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff564107620 0x7ff56406b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:56662/0 (socket says 172.21.15.27:56662) 2023-12-17T12:09:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.653+0000 7ff569993700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56406e660 0x7ff56406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.653+0000 7ff569993700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56406e660 0x7ff56406eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58584/0 (socket says 172.21.15.27:58584) 2023-12-17T12:09:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.653+0000 7ff569993700 1 -- 172.21.15.27:0/2334210969 learned_addr learned my addr 172.21.15.27:0/2334210969 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.654+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56406f050 con 0x7ff56406e660 2023-12-17T12:09:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.654+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56406f1c0 con 0x7ff564069930 2023-12-17T12:09:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.654+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56406f330 con 0x7ff564107620 2023-12-17T12:09:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff569192700 1 -- 172.21.15.27:0/2334210969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff564107620 msgr2=0x7ff56406b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff569192700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff564107620 0x7ff56406b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff569192700 1 -- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56406e660 msgr2=0x7ff56406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff569192700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56406e660 0x7ff56406eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff569192700 1 -- 172.21.15.27:0/2334210969 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff560009520 con 0x7ff564069930 2023-12-17T12:09:30.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff568991700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff564107620 0x7ff56406b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:30.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff569993700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56406e660 0x7ff56406eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:30.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff569192700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff564069930 0x7ff56406b1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9f0 tx=0x7ff56000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff55a7fc700 1 -- 172.21.15.27:0/2334210969 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56000cb70 con 0x7ff564069930 2023-12-17T12:09:30.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff55a7fc700 1 -- 172.21.15.27:0/2334210969 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff56000ee90 con 0x7ff564069930 2023-12-17T12:09:30.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff55a7fc700 1 -- 172.21.15.27:0/2334210969 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560003bc0 con 0x7ff564069930 2023-12-17T12:09:30.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.655+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5540095f0 con 0x7ff564069930 2023-12-17T12:09:30.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.656+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56406f8f0 con 0x7ff564069930 2023-12-17T12:09:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.658+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff564102cc0 con 0x7ff564069930 2023-12-17T12:09:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.659+0000 7ff55a7fc700 1 -- 172.21.15.27:0/2334210969 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff560003d60 con 0x7ff564069930 2023-12-17T12:09:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.661+0000 7ff55a7fc700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff5500a1590 0x7ff5500a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.661+0000 7ff55a7fc700 1 -- 172.21.15.27:0/2334210969 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff560014070 con 0x7ff564069930 2023-12-17T12:09:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.661+0000 7ff568991700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff5500a1590 0x7ff5500a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.662+0000 7ff568991700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff5500a1590 0x7ff5500a3a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff554009990 tx=0x7ff554005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.665+0000 7ff55a7fc700 1 -- 172.21.15.27:0/2334210969 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff560096560 con 0x7ff564069930 2023-12-17T12:09:30.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.614+0000 7f247c36b700 1 -- 172.21.15.27:0/2225908608 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2474069890 msgr2=0x7f247410dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.614+0000 7f247c36b700 1 --2- 172.21.15.27:0/2225908608 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2474069890 0x7f247410dba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2464009990 tx=0x7f2464009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.614+0000 7f247c36b700 1 -- 172.21.15.27:0/2225908608 shutdown_connections 2023-12-17T12:09:30.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.614+0000 7f247c36b700 1 --2- 172.21.15.27:0/2225908608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2474068f70 0x7f2474069350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.614+0000 7f247c36b700 1 --2- 172.21.15.27:0/2225908608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f247410e0e0 0x7f24741104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.614+0000 7f247c36b700 1 --2- 172.21.15.27:0/2225908608 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2474069890 0x7f247410dba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.614+0000 7f247c36b700 1 -- 172.21.15.27:0/2225908608 >> 172.21.15.27:0/2225908608 conn(0x7f2474076570 msgr2=0x7f2474076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.615+0000 7f247c36b700 1 -- 172.21.15.27:0/2225908608 shutdown_connections 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.615+0000 7f247c36b700 1 -- 172.21.15.27:0/2225908608 wait complete. 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f247c36b700 1 Processor -- start 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f247c36b700 1 -- start start 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f247c36b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2474068f70 0x7f247419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f247c36b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2474069890 0x7f247419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f247c36b700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f247410e0e0 0x7f24741999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f247c36b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247419a010 con 0x7f2474069890 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f247c36b700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247419a180 con 0x7f247410e0e0 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f247c36b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247419a340 con 0x7f2474068f70 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.616+0000 7f2479906700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2474069890 0x7f247419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f2479906700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2474069890 0x7f247419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58562/0 (socket says 172.21.15.27:58562) 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f2479906700 1 -- 172.21.15.27:0/1316148429 learned_addr learned my addr 172.21.15.27:0/1316148429 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:30.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f247a107700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2474068f70 0x7f247419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f2479906700 1 -- 172.21.15.27:0/1316148429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2474068f70 msgr2=0x7f247419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f2479906700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2474068f70 0x7f247419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f2479906700 1 -- 172.21.15.27:0/1316148429 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f247410e0e0 msgr2=0x7f24741999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f2479906700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f247410e0e0 0x7f24741999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f2479906700 1 -- 172.21.15.27:0/1316148429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24640095f0 con 0x7f2474069890 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.617+0000 7f2479906700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2474069890 0x7f247419fdb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2464004730 tx=0x7f2464004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.618+0000 7f246b7fe700 1 -- 172.21.15.27:0/1316148429 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246401d070 con 0x7f2474069890 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.618+0000 7f246b7fe700 1 -- 172.21.15.27:0/1316148429 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f246400be50 con 0x7f2474069890 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.618+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247419a5c0 con 0x7f2474069890 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.618+0000 7f246b7fe700 1 -- 172.21.15.27:0/1316148429 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464022e70 con 0x7f2474069890 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.618+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24741a6860 con 0x7f2474069890 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.620+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2474109300 con 0x7f2474069890 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.622+0000 7f246b7fe700 1 -- 172.21.15.27:0/1316148429 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f246400fac0 con 0x7f2474069890 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.623+0000 7f246b7fe700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f24600a1590 0x7f24600a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.624+0000 7f246b7fe700 1 -- 172.21.15.27:0/1316148429 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f24640ce120 con 0x7f2474069890 2023-12-17T12:09:30.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.624+0000 7f247a107700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f24600a1590 0x7f24600a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.625+0000 7f247a107700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f24600a1590 0x7f24600a3a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2470009c60 tx=0x7f2470009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.628+0000 7f246b7fe700 1 -- 172.21.15.27:0/1316148429 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f246405d2f0 con 0x7f2474069890 2023-12-17T12:09:30.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 -- 172.21.15.27:0/593620790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c1076f0 msgr2=0x7f073c107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 --2- 172.21.15.27:0/593620790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c1076f0 0x7f073c107ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f072c0098f0 tx=0x7f072c009c00 comp rx=0 tx=0).stop 2023-12-17T12:09:30.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 -- 172.21.15.27:0/593620790 shutdown_connections 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 --2- 172.21.15.27:0/593620790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073c102040 0x7f073c10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 --2- 172.21.15.27:0/593620790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c1076f0 0x7f073c107ad0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 --2- 172.21.15.27:0/593620790 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f073c101680 0x7f073c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 -- 172.21.15.27:0/593620790 >> 172.21.15.27:0/593620790 conn(0x7f073c0787c0 msgr2=0x7f073c078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 -- 172.21.15.27:0/593620790 shutdown_connections 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.414+0000 7f074244e700 1 -- 172.21.15.27:0/593620790 wait complete. 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.416+0000 7f074244e700 1 Processor -- start 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.416+0000 7f074244e700 1 -- start start 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f074244e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073c101680 0x7f073c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f074244e700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f073c102040 0x7f073c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f074244e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c1076f0 0x7f073c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f074244e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c19a050 con 0x7f073c1076f0 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f074244e700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c19a190 con 0x7f073c102040 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f074244e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c19a350 con 0x7f073c101680 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f073b7fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f073c102040 0x7f073c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f073b7fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f073c102040 0x7f073c19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:59118/0 (socket says 172.21.15.27:59118) 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.417+0000 7f073b7fe700 1 -- 172.21.15.27:0/1623472362 learned_addr learned my addr 172.21.15.27:0/1623472362 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.418+0000 7f073bfff700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073c101680 0x7f073c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.418+0000 7f073b7fe700 1 -- 172.21.15.27:0/1623472362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073c101680 msgr2=0x7f073c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.418+0000 7f073b7fe700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073c101680 0x7f073c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.418+0000 7f073b7fe700 1 -- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c1076f0 msgr2=0x7f073c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.418+0000 7f073b7fe700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c1076f0 0x7f073c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.418+0000 7f073b7fe700 1 -- 172.21.15.27:0/1623472362 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c009520 con 0x7f073c102040 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.418+0000 7f073bfff700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073c101680 0x7f073c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.418+0000 7f073b7fe700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f073c102040 0x7f073c19fd20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f073000e9a0 tx=0x7f073000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.419+0000 7f07397fa700 1 -- 172.21.15.27:0/1623472362 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730009790 con 0x7f073c102040 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.419+0000 7f07397fa700 1 -- 172.21.15.27:0/1623472362 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0730010430 con 0x7f073c102040 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.419+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c19a630 con 0x7f073c102040 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.419+0000 7f07397fa700 1 -- 172.21.15.27:0/1623472362 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730018680 con 0x7f073c102040 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.419+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c1a6880 con 0x7f073c102040 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.421+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c102d10 con 0x7f073c102040 2023-12-17T12:09:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.424+0000 7f07397fa700 1 -- 172.21.15.27:0/1623472362 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f073000cb50 con 0x7f073c102040 2023-12-17T12:09:30.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.426+0000 7f07397fa700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f07240a1610 0x7f07240a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.426+0000 7f07397fa700 1 -- 172.21.15.27:0/1623472362 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0730014070 con 0x7f073c102040 2023-12-17T12:09:30.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.426+0000 7f073bfff700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f07240a1610 0x7f07240a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.427+0000 7f073bfff700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f07240a1610 0x7f07240a3ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f072c009230 tx=0x7f072c005b40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.430+0000 7f07397fa700 1 -- 172.21.15.27:0/1623472362 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f07300522f0 con 0x7f073c102040 2023-12-17T12:09:30.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.726+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f073c04f290 con 0x7f073c102040 2023-12-17T12:09:30.765 INFO:teuthology.orchestra.run.smithi027.stdout:158913790000 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.727+0000 7f07397fa700 1 -- 172.21.15.27:0/1623472362 <== mon.2 v2:172.21.15.141:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0730096570 con 0x7f073c102040 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f07240a1610 msgr2=0x7f07240a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f07240a1610 0x7f07240a3ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f072c009230 tx=0x7f072c005b40 comp rx=0 tx=0).stop 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f073c102040 msgr2=0x7f073c19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f073c102040 0x7f073c19fd20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f073000e9a0 tx=0x7f073000ed60 comp rx=0 tx=0).stop 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 shutdown_connections 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f07240a1610 0x7f07240a3ad0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073c101680 0x7f073c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c1076f0 0x7f073c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.731+0000 7f074244e700 1 --2- 172.21.15.27:0/1623472362 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f073c102040 0x7f073c19fd20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.732+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 >> 172.21.15.27:0/1623472362 conn(0x7f073c0787c0 msgr2=0x7f073c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.732+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 shutdown_connections 2023-12-17T12:09:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.732+0000 7f074244e700 1 -- 172.21.15.27:0/1623472362 wait complete. 2023-12-17T12:09:30.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:30 smithi192 ceph-mon[41899]: pgmap v287: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3 op/s; 2/192 objects degraded (1.042%); 88 B/s, 0 keys/s, 0 objects/s recovering 2023-12-17T12:09:30.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.870+0000 7fba68cb4700 1 -- 172.21.15.27:0/2662480503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba64104030 msgr2=0x7fba641044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.870+0000 7fba68cb4700 1 --2- 172.21.15.27:0/2662480503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba64104030 0x7fba641044e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b0a0 tx=0x7fba5c00b3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.871+0000 7fba68cb4700 1 -- 172.21.15.27:0/2662480503 shutdown_connections 2023-12-17T12:09:30.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.871+0000 7fba68cb4700 1 --2- 172.21.15.27:0/2662480503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba64104030 0x7fba641044e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.871+0000 7fba68cb4700 1 --2- 172.21.15.27:0/2662480503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba641064d0 0x7fba64068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.871+0000 7fba68cb4700 1 --2- 172.21.15.27:0/2662480503 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fba640694b0 0x7fba64069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.871+0000 7fba68cb4700 1 -- 172.21.15.27:0/2662480503 >> 172.21.15.27:0/2662480503 conn(0x7fba64076570 msgr2=0x7fba64076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:30.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.871+0000 7fba68cb4700 1 -- 172.21.15.27:0/2662480503 shutdown_connections 2023-12-17T12:09:30.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.871+0000 7fba68cb4700 1 -- 172.21.15.27:0/2662480503 wait complete. 2023-12-17T12:09:30.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.872+0000 7fba68cb4700 1 Processor -- start 2023-12-17T12:09:30.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.872+0000 7fba68cb4700 1 -- start start 2023-12-17T12:09:30.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba68cb4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba640694b0 0x7fba6419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba68cb4700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fba64104030 0x7fba6419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba68cb4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba641064d0 0x7fba641a2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba68cb4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a2d50 con 0x7fba641064d0 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba68cb4700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a2ec0 con 0x7fba64104030 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba68cb4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a3080 con 0x7fba640694b0 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba62d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba641064d0 0x7fba641a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba61d9b700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fba64104030 0x7fba6419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba62d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba641064d0 0x7fba641a2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58606/0 (socket says 172.21.15.27:58606) 2023-12-17T12:09:30.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.873+0000 7fba62d9d700 1 -- 172.21.15.27:0/3808942788 learned_addr learned my addr 172.21.15.27:0/3808942788 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:30.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.874+0000 7fba6259c700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba640694b0 0x7fba6419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.874+0000 7fba62d9d700 1 -- 172.21.15.27:0/3808942788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba640694b0 msgr2=0x7fba6419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.874+0000 7fba62d9d700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba640694b0 0x7fba6419dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.874+0000 7fba62d9d700 1 -- 172.21.15.27:0/3808942788 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fba64104030 msgr2=0x7fba6419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.874+0000 7fba62d9d700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fba64104030 0x7fba6419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.874+0000 7fba62d9d700 1 -- 172.21.15.27:0/3808942788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c009d00 con 0x7fba641064d0 2023-12-17T12:09:30.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.874+0000 7fba62d9d700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba641064d0 0x7fba641a2740 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b070 tx=0x7fba5c008df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.875+0000 7fba5b7fe700 1 -- 172.21.15.27:0/3808942788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c00e040 con 0x7fba641064d0 2023-12-17T12:09:30.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.875+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba641a3300 con 0x7fba641064d0 2023-12-17T12:09:30.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.875+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba64106d20 con 0x7fba641064d0 2023-12-17T12:09:30.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.875+0000 7fba5b7fe700 1 -- 172.21.15.27:0/3808942788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba5c00be00 con 0x7fba641064d0 2023-12-17T12:09:30.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.875+0000 7fba5b7fe700 1 -- 172.21.15.27:0/3808942788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c01de20 con 0x7fba641064d0 2023-12-17T12:09:30.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.876+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba641a3490 con 0x7fba641064d0 2023-12-17T12:09:30.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.879+0000 7fba5b7fe700 1 -- 172.21.15.27:0/3808942788 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fba5c004c60 con 0x7fba641064d0 2023-12-17T12:09:30.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.881+0000 7fba5b7fe700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fba480a1610 0x7fba480a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.882+0000 7fba5b7fe700 1 -- 172.21.15.27:0/3808942788 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fba5c0ce020 con 0x7fba641064d0 2023-12-17T12:09:30.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.882+0000 7fba6259c700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fba480a1610 0x7fba480a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.884+0000 7fba6259c700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fba480a1610 0x7fba480a3ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba54009b10 tx=0x7fba54009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.885+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/1006931941 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff4a4107620 msgr2=0x7ff4a4107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.885+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/1006931941 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff4a4107620 0x7ff4a4107a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00099c0 tx=0x7ff4a0009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.885+0000 7fba5b7fe700 1 -- 172.21.15.27:0/3808942788 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fba5c0975e0 con 0x7fba641064d0 2023-12-17T12:09:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.885+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/1006931941 shutdown_connections 2023-12-17T12:09:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.885+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/1006931941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a4069930 0x7ff4a410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.885+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/1006931941 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff4a4107620 0x7ff4a4107a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.885+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/1006931941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a4068f70 0x7ff4a40693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.885+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/1006931941 >> 172.21.15.27:0/1006931941 conn(0x7ff4a4076570 msgr2=0x7ff4a4076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:30.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.887+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/1006931941 shutdown_connections 2023-12-17T12:09:30.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.887+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/1006931941 wait complete. 2023-12-17T12:09:30.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.888+0000 7ff4ac2c6700 1 Processor -- start 2023-12-17T12:09:30.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.888+0000 7ff4ac2c6700 1 -- start start 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.888+0000 7ff4ac2c6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a4068f70 0x7ff4a419f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.888+0000 7ff4ac2c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a4069930 0x7ff4a419fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4ac2c6700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff4a4107620 0x7ff4a4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4ac2c6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419a130 con 0x7ff4a4069930 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4ac2c6700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419a2a0 con 0x7ff4a4107620 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4ac2c6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419a460 con 0x7ff4a4068f70 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4a9861700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a4069930 0x7ff4a419fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4a9861700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a4069930 0x7ff4a419fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58634/0 (socket says 172.21.15.27:58634) 2023-12-17T12:09:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4a9861700 1 -- 172.21.15.27:0/850654026 learned_addr learned my addr 172.21.15.27:0/850654026 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4aa863700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff4a4107620 0x7ff4a4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.889+0000 7ff4a9861700 1 -- 172.21.15.27:0/850654026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a4068f70 msgr2=0x7ff4a419f990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.890+0000 7ff4a9861700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a4068f70 0x7ff4a419f990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.890+0000 7ff4a9861700 1 -- 172.21.15.27:0/850654026 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff4a4107620 msgr2=0x7ff4a4199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.890+0000 7ff4a9861700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff4a4107620 0x7ff4a4199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.890+0000 7ff4a9861700 1 -- 172.21.15.27:0/850654026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a00095f0 con 0x7ff4a4069930 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.890+0000 7ff4a9861700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a4069930 0x7ff4a419fed0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d980 tx=0x7ff49400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.890+0000 7ff49b7fe700 1 -- 172.21.15.27:0/850654026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494004220 con 0x7ff4a4069930 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.890+0000 7ff49b7fe700 1 -- 172.21.15.27:0/850654026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff494007640 con 0x7ff4a4069930 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.891+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a419a740 con 0x7ff4a4069930 2023-12-17T12:09:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.891+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a41a6980 con 0x7ff4a4069930 2023-12-17T12:09:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.891+0000 7ff49b7fe700 1 -- 172.21.15.27:0/850654026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940055e0 con 0x7ff4a4069930 2023-12-17T12:09:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.893+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4880052f0 con 0x7ff4a4069930 2023-12-17T12:09:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.896+0000 7ff49b7fe700 1 -- 172.21.15.27:0/850654026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff494004380 con 0x7ff4a4069930 2023-12-17T12:09:30.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.899+0000 7ff49b7fe700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff4900a1590 0x7ff4900a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:30.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.899+0000 7ff49b7fe700 1 -- 172.21.15.27:0/850654026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4940ccad0 con 0x7ff4a4069930 2023-12-17T12:09:30.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.899+0000 7ff4aa062700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff4900a1590 0x7ff4900a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:30.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.900+0000 7ff4aa062700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff4900a1590 0x7ff4900a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009990 tx=0x7ff4a000b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:30.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.902+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f247419ae70 con 0x7f2474069890 2023-12-17T12:09:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.903+0000 7f246b7fe700 1 -- 172.21.15.27:0/1316148429 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2464097660 con 0x7f2474069890 2023-12-17T12:09:30.904 INFO:teuthology.orchestra.run.smithi027.stdout:34359738437 2023-12-17T12:09:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.904+0000 7ff49b7fe700 1 -- 172.21.15.27:0/850654026 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff494099a70 con 0x7ff4a4069930 2023-12-17T12:09:30.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f24600a1590 msgr2=0x7f24600a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f24600a1590 0x7f24600a3a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2470009c60 tx=0x7f2470009260 comp rx=0 tx=0).stop 2023-12-17T12:09:30.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2474069890 msgr2=0x7f247419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2474069890 0x7f247419fdb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2464004730 tx=0x7f2464004760 comp rx=0 tx=0).stop 2023-12-17T12:09:30.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 shutdown_connections 2023-12-17T12:09:30.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f24600a1590 0x7f24600a3a50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2474068f70 0x7f247419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f247410e0e0 0x7f24741999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 --2- 172.21.15.27:0/1316148429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2474069890 0x7f247419fdb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.906+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 >> 172.21.15.27:0/1316148429 conn(0x7f2474076570 msgr2=0x7f247410f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.908+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 shutdown_connections 2023-12-17T12:09:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.908+0000 7f247c36b700 1 -- 172.21.15.27:0/1316148429 wait complete. 2023-12-17T12:09:30.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.927+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff564100860 con 0x7ff564069930 2023-12-17T12:09:30.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.928+0000 7ff55a7fc700 1 -- 172.21.15.27:0/2334210969 <== mon.2 v2:172.21.15.141:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff560096560 con 0x7ff564069930 2023-12-17T12:09:30.929 INFO:teuthology.orchestra.run.smithi027.stdout:137438953523 2023-12-17T12:09:30.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff5500a1590 msgr2=0x7ff5500a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff5500a1590 0x7ff5500a3a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff554009990 tx=0x7ff554005fb0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff564069930 msgr2=0x7ff56406b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff564069930 0x7ff56406b1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9f0 tx=0x7ff56000ed00 comp rx=0 tx=0).stop 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 shutdown_connections 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff5500a1590 0x7ff5500a3a50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56406e660 0x7ff56406eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff564069930 0x7ff56406b1c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 --2- 172.21.15.27:0/2334210969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff564107620 0x7ff56406b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 >> 172.21.15.27:0/2334210969 conn(0x7ff564076570 msgr2=0x7ff5640fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 shutdown_connections 2023-12-17T12:09:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:30.932+0000 7ff56b3f6700 1 -- 172.21.15.27:0/2334210969 wait complete. 2023-12-17T12:09:31.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:30 smithi141 ceph-mon[41225]: pgmap v287: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3 op/s; 2/192 objects degraded (1.042%); 88 B/s, 0 keys/s, 0 objects/s recovering 2023-12-17T12:09:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.130+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fba641a3490 con 0x7fba641064d0 2023-12-17T12:09:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.130+0000 7fba5b7fe700 1 -- 172.21.15.27:0/3808942788 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba641a3490 con 0x7fba641064d0 2023-12-17T12:09:31.132 INFO:teuthology.orchestra.run.smithi027.stdout:240518168609 2023-12-17T12:09:31.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fba480a1610 msgr2=0x7fba480a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fba480a1610 0x7fba480a3ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba54009b10 tx=0x7fba54009260 comp rx=0 tx=0).stop 2023-12-17T12:09:31.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba641064d0 msgr2=0x7fba641a2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba641064d0 0x7fba641a2740 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b070 tx=0x7fba5c008df0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 shutdown_connections 2023-12-17T12:09:31.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fba480a1610 0x7fba480a3ad0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba640694b0 0x7fba6419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba641064d0 0x7fba641a2740 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 --2- 172.21.15.27:0/3808942788 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fba64104030 0x7fba6419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 >> 172.21.15.27:0/3808942788 conn(0x7fba64076570 msgr2=0x7fba64101000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:31.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 shutdown_connections 2023-12-17T12:09:31.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.133+0000 7fba68cb4700 1 -- 172.21.15.27:0/3808942788 wait complete. 2023-12-17T12:09:31.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.150+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff488005160 con 0x7ff4a4069930 2023-12-17T12:09:31.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.150+0000 7ff49b7fe700 1 -- 172.21.15.27:0/850654026 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff494016020 con 0x7ff4a4069930 2023-12-17T12:09:31.152 INFO:teuthology.orchestra.run.smithi027.stdout:77309411388 2023-12-17T12:09:31.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff4900a1590 msgr2=0x7ff4900a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff4900a1590 0x7ff4900a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009990 tx=0x7ff4a000b540 comp rx=0 tx=0).stop 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a4069930 msgr2=0x7ff4a419fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a4069930 0x7ff4a419fed0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d980 tx=0x7ff49400dc90 comp rx=0 tx=0).stop 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 shutdown_connections 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff4900a1590 0x7ff4900a3a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a4068f70 0x7ff4a419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff4a4107620 0x7ff4a4199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 --2- 172.21.15.27:0/850654026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a4069930 0x7ff4a419fed0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 >> 172.21.15.27:0/850654026 conn(0x7ff4a4076570 msgr2=0x7ff4a40ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 shutdown_connections 2023-12-17T12:09:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.153+0000 7ff4ac2c6700 1 -- 172.21.15.27:0/850654026 wait complete. 2023-12-17T12:09:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 -- 172.21.15.27:0/921436009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c0074120 msgr2=0x7fe3c010daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 --2- 172.21.15.27:0/921436009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c0074120 0x7fe3c010daf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009990 tx=0x7fe3b0009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 -- 172.21.15.27:0/921436009 shutdown_connections 2023-12-17T12:09:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 --2- 172.21.15.27:0/921436009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3c010e030 0x7fe3c0110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 --2- 172.21.15.27:0/921436009 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe3c0073800 0x7fe3c0073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 --2- 172.21.15.27:0/921436009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c0074120 0x7fe3c010daf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 -- 172.21.15.27:0/921436009 >> 172.21.15.27:0/921436009 conn(0x7fe3c00787c0 msgr2=0x7fe3c0078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 -- 172.21.15.27:0/921436009 shutdown_connections 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.248+0000 7fe3c731d700 1 -- 172.21.15.27:0/921436009 wait complete. 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.250+0000 7fe3c731d700 1 Processor -- start 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.250+0000 7fe3c731d700 1 -- start start 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c731d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe3c0073800 0x7fe3c019f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c731d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3c0074120 0x7fe3c019fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c731d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c010e030 0x7fe3c0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c731d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c019a130 con 0x7fe3c010e030 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c731d700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c019a2a0 con 0x7fe3c0073800 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c731d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c019a460 con 0x7fe3c0074120 2023-12-17T12:09:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c58ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c010e030 0x7fe3c0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c58ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c010e030 0x7fe3c0199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58648/0 (socket says 172.21.15.27:58648) 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c58ba700 1 -- 172.21.15.27:0/3520482304 learned_addr learned my addr 172.21.15.27:0/3520482304 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c50b9700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe3c0073800 0x7fe3c019f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.251+0000 7fe3c48b8700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3c0074120 0x7fe3c019fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3c58ba700 1 -- 172.21.15.27:0/3520482304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3c0074120 msgr2=0x7fe3c019fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3c58ba700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3c0074120 0x7fe3c019fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3c58ba700 1 -- 172.21.15.27:0/3520482304 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe3c0073800 msgr2=0x7fe3c019f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3c58ba700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe3c0073800 0x7fe3c019f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3c58ba700 1 -- 172.21.15.27:0/3520482304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00095f0 con 0x7fe3c010e030 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3c58ba700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c010e030 0x7fe3c0199ad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800efc0 tx=0x7fe3b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3b67fc700 1 -- 172.21.15.27:0/3520482304 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8018070 con 0x7fe3c010e030 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c019a6e0 con 0x7fe3c010e030 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3b67fc700 1 -- 172.21.15.27:0/3520482304 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3b800f040 con 0x7fe3c010e030 2023-12-17T12:09:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3b67fc700 1 -- 172.21.15.27:0/3520482304 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800cb80 con 0x7fe3c010e030 2023-12-17T12:09:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.252+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c01a6980 con 0x7fe3c010e030 2023-12-17T12:09:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.254+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c0109240 con 0x7fe3c010e030 2023-12-17T12:09:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.257+0000 7fe3b67fc700 1 -- 172.21.15.27:0/3520482304 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe3b8003a00 con 0x7fe3c010e030 2023-12-17T12:09:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.259+0000 7fe3b67fc700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe3ac0a1510 0x7fe3ac0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.259+0000 7fe3b67fc700 1 -- 172.21.15.27:0/3520482304 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe3b80cd190 con 0x7fe3c010e030 2023-12-17T12:09:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.259+0000 7fe3c50b9700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe3ac0a1510 0x7fe3ac0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.260+0000 7fe3c50b9700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe3ac0a1510 0x7fe3ac0a39d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009c40 tx=0x7fe3bc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.261+0000 7fe3b67fc700 1 -- 172.21.15.27:0/3520482304 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe3b8096730 con 0x7fe3c010e030 2023-12-17T12:09:31.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.498+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe3c0074e90 con 0x7fe3c010e030 2023-12-17T12:09:31.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.499+0000 7fe3b67fc700 1 -- 172.21.15.27:0/3520482304 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe3b8003cb0 con 0x7fe3c010e030 2023-12-17T12:09:31.500 INFO:teuthology.orchestra.run.smithi027.stdout:55834574912 2023-12-17T12:09:31.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe3ac0a1510 msgr2=0x7fe3ac0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe3ac0a1510 0x7fe3ac0a39d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009c40 tx=0x7fe3bc009260 comp rx=0 tx=0).stop 2023-12-17T12:09:31.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c010e030 msgr2=0x7fe3c0199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c010e030 0x7fe3c0199ad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800efc0 tx=0x7fe3b800c3a0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 shutdown_connections 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fe3ac0a1510 0x7fe3ac0a39d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c010e030 0x7fe3c0199ad0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe3c0073800 0x7fe3c019f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 --2- 172.21.15.27:0/3520482304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3c0074120 0x7fe3c019fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 >> 172.21.15.27:0/3520482304 conn(0x7fe3c00787c0 msgr2=0x7fe3c00ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 shutdown_connections 2023-12-17T12:09:31.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:31.501+0000 7fe3c731d700 1 -- 172.21.15.27:0/3520482304 wait complete. 2023-12-17T12:09:31.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:31 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1623472362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-17T12:09:31.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:31 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1316148429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-17T12:09:31.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:31 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2334210969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-17T12:09:31.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:31 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3808942788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-17T12:09:31.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:31 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/850654026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-17T12:09:31.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:31 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3520482304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-17T12:09:31.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:31 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1623472362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-17T12:09:31.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:31 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1316148429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-17T12:09:31.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:31 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2334210969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-17T12:09:31.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:31 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3808942788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-17T12:09:31.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:31 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/850654026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-17T12:09:31.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:31 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3520482304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-17T12:09:32.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:31 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1623472362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-17T12:09:32.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:31 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1316148429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-17T12:09:32.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:31 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2334210969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-17T12:09:32.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:31 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3808942788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-17T12:09:32.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:31 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/850654026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-17T12:09:32.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:31 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3520482304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-17T12:09:32.084 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411385 got 77309411388 for osd.2 2023-12-17T12:09:32.084 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:32.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953521 got 137438953523 for osd.4 2023-12-17T12:09:32.268 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:32.416 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168607 got 240518168609 for osd.8 2023-12-17T12:09:32.416 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:32.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:32 smithi027 ceph-mon[36871]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 7.1 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:32.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:32 smithi027 ceph-mon[36871]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded) 2023-12-17T12:09:32.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:32 smithi027 ceph-mon[36871]: Cluster is now healthy 2023-12-17T12:09:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:32 smithi192 ceph-mon[41899]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 7.1 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:32 smithi192 ceph-mon[41899]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded) 2023-12-17T12:09:32.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:32 smithi192 ceph-mon[41899]: Cluster is now healthy 2023-12-17T12:09:32.817 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574911 got 55834574912 for osd.1 2023-12-17T12:09:32.817 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:32.929 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:33.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:32 smithi141 ceph-mon[41225]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 7.1 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:33.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:32 smithi141 ceph-mon[41225]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded) 2023-12-17T12:09:33.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:32 smithi141 ceph-mon[41225]: Cluster is now healthy 2023-12-17T12:09:33.213 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789996 got 158913790000 for osd.5 2023-12-17T12:09:33.214 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:33.242 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738435 got 34359738437 for osd.0 2023-12-17T12:09:33.242 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:33.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 -- 172.21.15.27:0/4093951722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c107620 msgr2=0x7f091c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:33.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 --2- 172.21.15.27:0/4093951722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c107620 0x7f091c107a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f090c0099c0 tx=0x7f090c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 -- 172.21.15.27:0/4093951722 shutdown_connections 2023-12-17T12:09:33.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 --2- 172.21.15.27:0/4093951722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f091c069930 0x7f091c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 --2- 172.21.15.27:0/4093951722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c107620 0x7f091c107a00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 --2- 172.21.15.27:0/4093951722 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f091c068f70 0x7f091c0693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 -- 172.21.15.27:0/4093951722 >> 172.21.15.27:0/4093951722 conn(0x7f091c076570 msgr2=0x7f091c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:33.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 -- 172.21.15.27:0/4093951722 shutdown_connections 2023-12-17T12:09:33.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.405+0000 7f09224c3700 1 -- 172.21.15.27:0/4093951722 wait complete. 2023-12-17T12:09:33.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.406+0000 7f09224c3700 1 Processor -- start 2023-12-17T12:09:33.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.406+0000 7f09224c3700 1 -- start start 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f09224c3700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f091c068f70 0x7f091c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f09224c3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c069930 0x7f091c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f09224c3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f091c107620 0x7f091c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f09224c3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c199ff0 con 0x7f091c069930 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f09224c3700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c19a160 con 0x7f091c068f70 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f09224c3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c19a320 con 0x7f091c107620 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c069930 0x7f091c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c069930 0x7f091c19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58660/0 (socket says 172.21.15.27:58660) 2023-12-17T12:09:33.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f091b7fe700 1 -- 172.21.15.27:0/1997803638 learned_addr learned my addr 172.21.15.27:0/1997803638 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f091bfff700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f091c068f70 0x7f091c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f0920a60700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f091c107620 0x7f091c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.407+0000 7f091b7fe700 1 -- 172.21.15.27:0/1997803638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f091c107620 msgr2=0x7f091c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f091b7fe700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f091c107620 0x7f091c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f091b7fe700 1 -- 172.21.15.27:0/1997803638 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f091c068f70 msgr2=0x7f091c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f091b7fe700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f091c068f70 0x7f091c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f091b7fe700 1 -- 172.21.15.27:0/1997803638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c0095f0 con 0x7f091c069930 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f091bfff700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f091c068f70 0x7f091c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f091b7fe700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c069930 0x7f091c19fd90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f091000d950 tx=0x7f091000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f09197fa700 1 -- 172.21.15.27:0/1997803638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0910009b40 con 0x7f091c069930 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f09197fa700 1 -- 172.21.15.27:0/1997803638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0910009ca0 con 0x7f091c069930 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c19a600 con 0x7f091c069930 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f09197fa700 1 -- 172.21.15.27:0/1997803638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091000f620 con 0x7f091c069930 2023-12-17T12:09:33.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.408+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c1a6840 con 0x7f091c069930 2023-12-17T12:09:33.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.410+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c102cc0 con 0x7f091c069930 2023-12-17T12:09:33.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.411+0000 7f09197fa700 1 -- 172.21.15.27:0/1997803638 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f091000f780 con 0x7f091c069930 2023-12-17T12:09:33.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.413+0000 7f09197fa700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09040a1590 0x7f09040a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:33.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.413+0000 7f09197fa700 1 -- 172.21.15.27:0/1997803638 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f09100cd1d0 con 0x7f091c069930 2023-12-17T12:09:33.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.413+0000 7f091bfff700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09040a1590 0x7f09040a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:33.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.414+0000 7f091bfff700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09040a1590 0x7f09040a3a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f090c009f50 tx=0x7f090c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:33.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.417+0000 7f09197fa700 1 -- 172.21.15.27:0/1997803638 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0910096680 con 0x7f091c069930 2023-12-17T12:09:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.658+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f091c19ae80 con 0x7f091c069930 2023-12-17T12:09:33.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.658+0000 7f09197fa700 1 -- 172.21.15.27:0/1997803638 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f0910099ca0 con 0x7f091c069930 2023-12-17T12:09:33.660 INFO:teuthology.orchestra.run.smithi027.stdout:296352743448 2023-12-17T12:09:33.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09040a1590 msgr2=0x7f09040a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09040a1590 0x7f09040a3a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f090c009f50 tx=0x7f090c005fb0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c069930 msgr2=0x7f091c19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c069930 0x7f091c19fd90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f091000d950 tx=0x7f091000dd10 comp rx=0 tx=0).stop 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 shutdown_connections 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f09040a1590 0x7f09040a3a50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f091c107620 0x7f091c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f091c068f70 0x7f091c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 --2- 172.21.15.27:0/1997803638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c069930 0x7f091c19fd90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 >> 172.21.15.27:0/1997803638 conn(0x7f091c076570 msgr2=0x7f091c10c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 shutdown_connections 2023-12-17T12:09:33.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.661+0000 7f09224c3700 1 -- 172.21.15.27:0/1997803638 wait complete. 2023-12-17T12:09:33.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 -- 172.21.15.27:0/2075210984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c101680 msgr2=0x7f2f2c101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:33.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/2075210984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c101680 0x7f2f2c101b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009990 tx=0x7f2f18009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 -- 172.21.15.27:0/2075210984 shutdown_connections 2023-12-17T12:09:33.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/2075210984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f2c102040 0x7f2f2c10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/2075210984 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2f2c1076f0 0x7f2f2c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/2075210984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c101680 0x7f2f2c101b00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 -- 172.21.15.27:0/2075210984 >> 172.21.15.27:0/2075210984 conn(0x7f2f2c0787c0 msgr2=0x7f2f2c078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:33.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 -- 172.21.15.27:0/2075210984 shutdown_connections 2023-12-17T12:09:33.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.765+0000 7f2f32fe0700 1 -- 172.21.15.27:0/2075210984 wait complete. 2023-12-17T12:09:33.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.766+0000 7f2f32fe0700 1 Processor -- start 2023-12-17T12:09:33.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f32fe0700 1 -- start start 2023-12-17T12:09:33.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f32fe0700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2f2c101680 0x7f2f2c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:33.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f32fe0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c102040 0x7f2f2c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:33.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f32fe0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f2c1076f0 0x7f2f2c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:33.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f32fe0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c199fb0 con 0x7f2f2c102040 2023-12-17T12:09:33.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f32fe0700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c19a120 con 0x7f2f2c101680 2023-12-17T12:09:33.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f32fe0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c19a2e0 con 0x7f2f2c1076f0 2023-12-17T12:09:33.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f2bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c102040 0x7f2f2c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:33.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f2bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c102040 0x7f2f2c19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58692/0 (socket says 172.21.15.27:58692) 2023-12-17T12:09:33.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f2bfff700 1 -- 172.21.15.27:0/3981245952 learned_addr learned my addr 172.21.15.27:0/3981245952 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:33.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.767+0000 7f2f3157d700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f2c1076f0 0x7f2f2c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:33.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f30d7c700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2f2c101680 0x7f2f2c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f2bfff700 1 -- 172.21.15.27:0/3981245952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f2c1076f0 msgr2=0x7f2f2c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f2bfff700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f2c1076f0 0x7f2f2c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f2bfff700 1 -- 172.21.15.27:0/3981245952 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2f2c101680 msgr2=0x7f2f2c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f2bfff700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2f2c101680 0x7f2f2c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f2bfff700 1 -- 172.21.15.27:0/3981245952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f180095f0 con 0x7f2f2c102040 2023-12-17T12:09:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f30d7c700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2f2c101680 0x7f2f2c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f2bfff700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c102040 0x7f2f2c19fd50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2f18004730 tx=0x7f2f18004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.768+0000 7f2f29ffb700 1 -- 172.21.15.27:0/3981245952 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1801d070 con 0x7f2f2c102040 2023-12-17T12:09:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.769+0000 7f2f29ffb700 1 -- 172.21.15.27:0/3981245952 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f18022470 con 0x7f2f2c102040 2023-12-17T12:09:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.769+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c19a560 con 0x7f2f2c102040 2023-12-17T12:09:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.769+0000 7f2f29ffb700 1 -- 172.21.15.27:0/3981245952 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18017650 con 0x7f2f2c102040 2023-12-17T12:09:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.769+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c1a6800 con 0x7f2f2c102040 2023-12-17T12:09:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.770+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c104940 con 0x7f2f2c102040 2023-12-17T12:09:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.772+0000 7f2f29ffb700 1 -- 172.21.15.27:0/3981245952 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2f180225e0 con 0x7f2f2c102040 2023-12-17T12:09:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.774+0000 7f2f29ffb700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.774+0000 7f2f29ffb700 1 -- 172.21.15.27:0/3981245952 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f180ce190 con 0x7f2f2c102040 2023-12-17T12:09:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.774+0000 7f2f30d7c700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:33.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.775+0000 7f2f30d7c700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000bc10 tx=0x7f2f2000b5f0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:33.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:33.779+0000 7f2f29ffb700 1 -- 172.21.15.27:0/3981245952 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2f180977e0 con 0x7f2f2c102040 2023-12-17T12:09:34.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.007+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2f2c04f290 con 0x7f2f2c102040 2023-12-17T12:09:34.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.007+0000 7f2f29ffb700 1 -- 172.21.15.27:0/3981245952 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f1809ae00 con 0x7f2f2c102040 2023-12-17T12:09:34.008 INFO:teuthology.orchestra.run.smithi027.stdout:210453397543 2023-12-17T12:09:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2f1c0a1510 msgr2=0x7f2f1c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000bc10 tx=0x7f2f2000b5f0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c102040 msgr2=0x7f2f2c19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c102040 0x7f2f2c19fd50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2f18004730 tx=0x7f2f18004760 comp rx=0 tx=0).stop 2023-12-17T12:09:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 shutdown_connections 2023-12-17T12:09:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f2c1076f0 0x7f2f2c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2f2c101680 0x7f2f2c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 --2- 172.21.15.27:0/3981245952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c102040 0x7f2f2c19fd50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 >> 172.21.15.27:0/3981245952 conn(0x7f2f2c0787c0 msgr2=0x7f2f2c10cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 shutdown_connections 2023-12-17T12:09:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.010+0000 7f2f32fe0700 1 -- 172.21.15.27:0/3981245952 wait complete. 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.034+0000 7fc4bb241700 1 -- 172.21.15.27:0/3542229186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 msgr2=0x7fc4b410e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.034+0000 7fc4bb241700 1 --2- 172.21.15.27:0/3542229186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 0x7fc4b410e4b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009f90 tx=0x7fc4ac00b330 comp rx=0 tx=0).stop 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.035+0000 7fc4bb241700 1 -- 172.21.15.27:0/3542229186 shutdown_connections 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.035+0000 7fc4bb241700 1 --2- 172.21.15.27:0/3542229186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 0x7fc4b410e4b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.035+0000 7fc4bb241700 1 --2- 172.21.15.27:0/3542229186 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc4b40691d0 0x7fc4b40695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.035+0000 7fc4bb241700 1 --2- 172.21.15.27:0/3542229186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b4069b80 0x7fc4b41061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.035+0000 7fc4bb241700 1 -- 172.21.15.27:0/3542229186 >> 172.21.15.27:0/3542229186 conn(0x7fc4b4076570 msgr2=0x7fc4b4076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.035+0000 7fc4bb241700 1 -- 172.21.15.27:0/3542229186 shutdown_connections 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.035+0000 7fc4bb241700 1 -- 172.21.15.27:0/3542229186 wait complete. 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.036+0000 7fc4bb241700 1 Processor -- start 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.037+0000 7fc4bb241700 1 -- start start 2023-12-17T12:09:34.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.037+0000 7fc4bb241700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b40691d0 0x7fc4b410c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.037+0000 7fc4bb241700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc4b4069b80 0x7fc4b410cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.037+0000 7fc4bb241700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 0x7fc4b41a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.037+0000 7fc4bb241700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41a2f60 con 0x7fc4b4106890 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.037+0000 7fc4bb241700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41a30d0 con 0x7fc4b4069b80 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.037+0000 7fc4bb241700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41a3290 con 0x7fc4b40691d0 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 0x7fc4b41a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b3fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc4b4069b80 0x7fc4b410cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 0x7fc4b41a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58704/0 (socket says 172.21.15.27:58704) 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 -- 172.21.15.27:0/231230764 learned_addr learned my addr 172.21.15.27:0/231230764 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b8fdd700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b40691d0 0x7fc4b410c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 -- 172.21.15.27:0/231230764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b40691d0 msgr2=0x7fc4b410c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b40691d0 0x7fc4b410c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 -- 172.21.15.27:0/231230764 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc4b4069b80 msgr2=0x7fc4b410cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc4b4069b80 0x7fc4b410cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 -- 172.21.15.27:0/231230764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a80095f0 con 0x7fc4b4106890 2023-12-17T12:09:34.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b8fdd700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b40691d0 0x7fc4b410c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.038+0000 7fc4b97de700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 0x7fc4b41a2950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00cc30 tx=0x7fc4ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.039+0000 7fc4b1ffb700 1 -- 172.21.15.27:0/231230764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00e040 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.039+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac009c40 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.039+0000 7fc4b1ffb700 1 -- 172.21.15.27:0/231230764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4ac00ceb0 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.039+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b41aac00 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.040+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b4102ce0 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.040+0000 7fc4b1ffb700 1 -- 172.21.15.27:0/231230764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac01d680 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.044+0000 7fc4b1ffb700 1 -- 172.21.15.27:0/231230764 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fc4ac005040 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.045+0000 7fc4b1ffb700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc4a40a1510 0x7fc4a40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.045+0000 7fc4b1ffb700 1 -- 172.21.15.27:0/231230764 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc4ac0ced50 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.046+0000 7fc4b8fdd700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc4a40a1510 0x7fc4a40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.046+0000 7fc4b1ffb700 1 -- 172.21.15.27:0/231230764 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc4ac098410 con 0x7fc4b4106890 2023-12-17T12:09:34.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.046+0000 7fc4b8fdd700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc4a40a1510 0x7fc4a40a39d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8005fd0 tx=0x7fc4a8009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:34.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.271+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc4b4111400 con 0x7fc4b4106890 2023-12-17T12:09:34.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.271+0000 7fc4b1ffb700 1 -- 172.21.15.27:0/231230764 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc4ac021020 con 0x7fc4b4106890 2023-12-17T12:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout:188978561068 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.273+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc4a40a1510 msgr2=0x7fc4a40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.273+0000 7fc4bb241700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc4a40a1510 0x7fc4a40a39d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8005fd0 tx=0x7fc4a8009310 comp rx=0 tx=0).stop 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 msgr2=0x7fc4b41a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 0x7fc4b41a2950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00cc30 tx=0x7fc4ac00cc60 comp rx=0 tx=0).stop 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 shutdown_connections 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fc4a40a1510 0x7fc4a40a39d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b40691d0 0x7fc4b410c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b4106890 0x7fc4b41a2950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 --2- 172.21.15.27:0/231230764 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc4b4069b80 0x7fc4b410cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 >> 172.21.15.27:0/231230764 conn(0x7fc4b4076570 msgr2=0x7fc4b4105210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 shutdown_connections 2023-12-17T12:09:34.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:34.274+0000 7fc4bb241700 1 -- 172.21.15.27:0/231230764 wait complete. 2023-12-17T12:09:34.370 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743446 got 296352743448 for osd.10 2023-12-17T12:09:34.370 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:34.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:34 smithi027 ceph-mon[36871]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 6.1 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:34.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:34 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1997803638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-17T12:09:34.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:34 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3981245952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-17T12:09:34.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:34 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/231230764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-17T12:09:34.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397540 got 210453397543 for osd.7 2023-12-17T12:09:34.807 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:34.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:34 smithi192 ceph-mon[41899]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 6.1 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:34.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:34 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1997803638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-17T12:09:34.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:34 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3981245952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-17T12:09:34.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:34 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/231230764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-17T12:09:34.933 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561066 got 188978561068 for osd.6 2023-12-17T12:09:34.933 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:35.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:34 smithi141 ceph-mon[41225]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 6.1 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:35.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:34 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1997803638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-17T12:09:35.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:34 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3981245952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-17T12:09:35.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:34 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/231230764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 -- 172.21.15.27:0/4095763315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0073800 msgr2=0x7f51a0073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 --2- 172.21.15.27:0/4095763315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0073800 0x7f51a0073be0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5190009990 tx=0x7f5190009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 -- 172.21.15.27:0/4095763315 shutdown_connections 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 --2- 172.21.15.27:0/4095763315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51a010e060 0x7f51a0110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 --2- 172.21.15.27:0/4095763315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0073800 0x7f51a0073be0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 --2- 172.21.15.27:0/4095763315 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51a0074120 0x7f51a010db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 -- 172.21.15.27:0/4095763315 >> 172.21.15.27:0/4095763315 conn(0x7f51a00786f0 msgr2=0x7f51a0078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 -- 172.21.15.27:0/4095763315 shutdown_connections 2023-12-17T12:09:35.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.650+0000 7f51a6489700 1 -- 172.21.15.27:0/4095763315 wait complete. 2023-12-17T12:09:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.651+0000 7f51a6489700 1 Processor -- start 2023-12-17T12:09:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.651+0000 7f51a6489700 1 -- start start 2023-12-17T12:09:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f51a6489700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51a0073800 0x7f51a019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f51a6489700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0074120 0x7f51a019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f51a6489700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51a010e060 0x7f51a0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f51a6489700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0199fc0 con 0x7f51a0074120 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f51a6489700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a019a130 con 0x7f51a0073800 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f51a6489700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a019a2f0 con 0x7f51a010e060 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f519f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0074120 0x7f51a019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f519ffff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51a0073800 0x7f51a019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f519f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0074120 0x7f51a019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58716/0 (socket says 172.21.15.27:58716) 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f519f7fe700 1 -- 172.21.15.27:0/4195139001 learned_addr learned my addr 172.21.15.27:0/4195139001 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f51a4a26700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51a010e060 0x7f51a0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f519f7fe700 1 -- 172.21.15.27:0/4195139001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51a010e060 msgr2=0x7f51a0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f519f7fe700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51a010e060 0x7f51a0199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f519f7fe700 1 -- 172.21.15.27:0/4195139001 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51a0073800 msgr2=0x7f51a019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.652+0000 7f519f7fe700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51a0073800 0x7f51a019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.653+0000 7f519f7fe700 1 -- 172.21.15.27:0/4195139001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900095f0 con 0x7f51a0074120 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.653+0000 7f51a4a26700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51a010e060 0x7f51a0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:35.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.653+0000 7f519f7fe700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0074120 0x7f51a019fd60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f519400e9a0 tx=0x7f519400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:35.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.653+0000 7f519d7fa700 1 -- 172.21.15.27:0/4195139001 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194009790 con 0x7f51a0074120 2023-12-17T12:09:35.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.653+0000 7f519d7fa700 1 -- 172.21.15.27:0/4195139001 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f519400cbd0 con 0x7f51a0074120 2023-12-17T12:09:35.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.653+0000 7f519d7fa700 1 -- 172.21.15.27:0/4195139001 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194018680 con 0x7f51a0074120 2023-12-17T12:09:35.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.653+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a019a5d0 con 0x7f51a0074120 2023-12-17T12:09:35.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.653+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a01a6810 con 0x7f51a0074120 2023-12-17T12:09:35.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.654+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a0109270 con 0x7f51a0074120 2023-12-17T12:09:35.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.656+0000 7f519d7fa700 1 -- 172.21.15.27:0/4195139001 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f519400cd40 con 0x7f51a0074120 2023-12-17T12:09:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.659+0000 7f519d7fa700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51880a1590 0x7f51880a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.659+0000 7f519d7fa700 1 -- 172.21.15.27:0/4195139001 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5194014070 con 0x7f51a0074120 2023-12-17T12:09:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.659+0000 7f519ffff700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51880a1590 0x7f51880a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.659+0000 7f519ffff700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51880a1590 0x7f51880a3a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5190006010 tx=0x7f519000b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:35.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.663+0000 7f519d7fa700 1 -- 172.21.15.27:0/4195139001 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f51940522f0 con 0x7f51a0074120 2023-12-17T12:09:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.874+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f51a0074e20 con 0x7f51a0074120 2023-12-17T12:09:35.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.874+0000 7f519d7fa700 1 -- 172.21.15.27:0/4195139001 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51940965a0 con 0x7f51a0074120 2023-12-17T12:09:35.876 INFO:teuthology.orchestra.run.smithi027.stdout:274877906973 2023-12-17T12:09:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.876+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51880a1590 msgr2=0x7f51880a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.876+0000 7f51a6489700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51880a1590 0x7f51880a3a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5190006010 tx=0x7f519000b260 comp rx=0 tx=0).stop 2023-12-17T12:09:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.876+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0074120 msgr2=0x7f51a019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.876+0000 7f51a6489700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0074120 0x7f51a019fd60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f519400e9a0 tx=0x7f519400ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.877+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 shutdown_connections 2023-12-17T12:09:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.877+0000 7f51a6489700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51880a1590 0x7f51880a3a50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.877+0000 7f51a6489700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51a010e060 0x7f51a0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.877+0000 7f51a6489700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51a0073800 0x7f51a019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.877+0000 7f51a6489700 1 --2- 172.21.15.27:0/4195139001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a0074120 0x7f51a019fd60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.877+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 >> 172.21.15.27:0/4195139001 conn(0x7f51a00786f0 msgr2=0x7f51a00fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.877+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 shutdown_connections 2023-12-17T12:09:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:35.877+0000 7f51a6489700 1 -- 172.21.15.27:0/4195139001 wait complete. 2023-12-17T12:09:36.437 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906973 for osd.9 2023-12-17T12:09:36.438 DEBUG:teuthology.parallel:result is None 2023-12-17T12:09:36.438 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-17T12:09:36.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-17T12:09:36.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:36 smithi192 ceph-mon[41899]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:36.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:36 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4195139001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-17T12:09:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:36.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:36 smithi027 ceph-mon[36871]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:36.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:36 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4195139001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-17T12:09:37.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:36 smithi141 ceph-mon[41225]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:37.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:36 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4195139001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-17T12:09:38.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 -- 172.21.15.27:0/2357277338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 msgr2=0x7f21b410e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:38.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2357277338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 0x7f21b410e430 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00b0a0 tx=0x7f21ac00b3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 -- 172.21.15.27:0/2357277338 shutdown_connections 2023-12-17T12:09:38.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2357277338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21b41076f0 0x7f21b4107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2357277338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 0x7f21b410e430 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2357277338 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f21b4101680 0x7f21b4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 -- 172.21.15.27:0/2357277338 >> 172.21.15.27:0/2357277338 conn(0x7f21b40787c0 msgr2=0x7f21b4078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:38.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 -- 172.21.15.27:0/2357277338 shutdown_connections 2023-12-17T12:09:38.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.224+0000 7f21bbd31700 1 -- 172.21.15.27:0/2357277338 wait complete. 2023-12-17T12:09:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.226+0000 7f21bbd31700 1 Processor -- start 2023-12-17T12:09:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.226+0000 7f21bbd31700 1 -- start start 2023-12-17T12:09:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21bbd31700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f21b4101680 0x7f21b419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21bbd31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 0x7f21b419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21bbd31700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21b41076f0 0x7f21b41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21bbd31700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b419a050 con 0x7f21b4102040 2023-12-17T12:09:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21bbd31700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b419a1c0 con 0x7f21b4101680 2023-12-17T12:09:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21bbd31700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b419a380 con 0x7f21b41076f0 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b92cc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 0x7f21b419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b92cc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 0x7f21b419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43640/0 (socket says 172.21.15.27:43640) 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b92cc700 1 -- 172.21.15.27:0/2514449314 learned_addr learned my addr 172.21.15.27:0/2514449314 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21ba2ce700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21b41076f0 0x7f21b41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b9acd700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f21b4101680 0x7f21b419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b92cc700 1 -- 172.21.15.27:0/2514449314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21b41076f0 msgr2=0x7f21b41999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b92cc700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21b41076f0 0x7f21b41999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b92cc700 1 -- 172.21.15.27:0/2514449314 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f21b4101680 msgr2=0x7f21b419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b92cc700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f21b4101680 0x7f21b419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.227+0000 7f21b92cc700 1 -- 172.21.15.27:0/2514449314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac009d00 con 0x7f21b4102040 2023-12-17T12:09:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.228+0000 7f21b92cc700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 0x7f21b419fdf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f21a400e9a0 tx=0x7f21a400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.228+0000 7f21aaffd700 1 -- 172.21.15.27:0/2514449314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4009790 con 0x7f21b4102040 2023-12-17T12:09:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.228+0000 7f21aaffd700 1 -- 172.21.15.27:0/2514449314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21a400cbd0 con 0x7f21b4102040 2023-12-17T12:09:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.228+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b419a660 con 0x7f21b4102040 2023-12-17T12:09:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.228+0000 7f21aaffd700 1 -- 172.21.15.27:0/2514449314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4018680 con 0x7f21b4102040 2023-12-17T12:09:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.228+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b41a68a0 con 0x7f21b4102040 2023-12-17T12:09:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.229+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b4104940 con 0x7f21b4102040 2023-12-17T12:09:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.231+0000 7f21aaffd700 1 -- 172.21.15.27:0/2514449314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f21a400cd40 con 0x7f21b4102040 2023-12-17T12:09:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.233+0000 7f21aaffd700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f21a00a1510 0x7f21a00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.233+0000 7f21aaffd700 1 -- 172.21.15.27:0/2514449314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f21a4014070 con 0x7f21b4102040 2023-12-17T12:09:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.233+0000 7f21b9acd700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f21a00a1510 0x7f21a00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:38.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.233+0000 7f21b9acd700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f21a00a1510 0x7f21a00a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009bf0 tx=0x7f21b0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:38.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.234+0000 7f21aaffd700 1 -- 172.21.15.27:0/2514449314 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f21a40968c0 con 0x7f21b4102040 2023-12-17T12:09:38.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.440+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21b419aeb0 con 0x7f21a00a1510 2023-12-17T12:09:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.452+0000 7f21aaffd700 1 -- 172.21.15.27:0/2514449314 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339417 (secure 0 0 0) 0x7f21b419aeb0 con 0x7f21a00a1510 2023-12-17T12:09:38.460 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f21a00a1510 msgr2=0x7f21a00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f21a00a1510 0x7f21a00a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009bf0 tx=0x7f21b0009260 comp rx=0 tx=0).stop 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 msgr2=0x7f21b419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 0x7f21b419fdf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f21a400e9a0 tx=0x7f21a400ed60 comp rx=0 tx=0).stop 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 shutdown_connections 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f21a00a1510 0x7f21a00a39d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21b41076f0 0x7f21b41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f21b4101680 0x7f21b419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 --2- 172.21.15.27:0/2514449314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21b4102040 0x7f21b419fdf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 >> 172.21.15.27:0/2514449314 conn(0x7f21b40787c0 msgr2=0x7f21b410caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 shutdown_connections 2023-12-17T12:09:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:38.458+0000 7f21bbd31700 1 -- 172.21.15.27:0/2514449314 wait complete. 2023-12-17T12:09:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-17T12:09:38.804 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:38 smithi027 ceph-mon[36871]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:38.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:38 smithi192 ceph-mon[41899]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-12-17T12:09:36.806626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652417,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":392,"num_read_kb":312,"num_write":330,"num_write_kb":3041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1239245,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335388,"kb_used_data":11128,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606436,"statfs":{"total":1151940427776,"available":1151596990464,"internally_reserved":0,"allocated":11395072,"data_stored":7251608,"data_compressed":116306,"data_compressed_allocated":147456,"data_compressed_original":1822720,"omap_allocated":19139,"internal_metadata":331855165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":33,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58509,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515969+0000","last_change":"2023-12-17T12:08:49.184999+0000","last_active":"2023-12-17T12:09:23.515969+0000","last_peered":"2023-12-17T12:09:23.515969+0000","last_clean":"2023-12-17T12:09:23.515969+0000","last_became_active":"2023-12-17T12:08:49.184214+0000","last_became_peered":"2023-12-17T12:08:49.184214+0000","last_unstale":"2023-12-17T12:09:23.515969+0000","last_undegraded":"2023-12-17T12:09:23.515969+0000","last_fullsized":"2023-12-17T12:09:23.515969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:48:56.174610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518792+0000","last_change":"2023-12-17T12:08:49.188212+0000","last_active":"2023-12-17T12:09:23.518792+0000","last_peered":"2023-12-17T12:09:23.518792+0000","last_clean":"2023-12-17T12:09:23.518792+0000","last_became_active":"2023-12-17T12:08:49.187869+0000","last_became_peered":"2023-12-17T12:08:49.187869+0000","last_unstale":"2023-12-17T12:09:23.518792+0000","last_undegraded":"2023-12-17T12:09:23.518792+0000","last_fullsized":"2023-12-17T12:09:23.518792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:53:52.485169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519450+0000","last_change":"2023-12-17T12:08:49.182801+0000","last_active":"2023-12-17T12:09:23.519450+0000","last_peered":"2023-12-17T12:09:23.519450+0000","last_clean":"2023-12-17T12:09:23.519450+0000","last_became_active":"2023-12-17T12:08:49.182488+0000","last_became_peered":"2023-12-17T12:08:49.182488+0000","last_unstale":"2023-12-17T12:09:23.519450+0000","last_undegraded":"2023-12-17T12:09:23.519450+0000","last_fullsized":"2023-12-17T12:09:23.519450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:16:43.566881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517285+0000","last_change":"2023-12-17T12:08:49.189279+0000","last_active":"2023-12-17T12:09:23.517285+0000","last_peered":"2023-12-17T12:09:23.517285+0000","last_clean":"2023-12-17T12:09:23.517285+0000","last_became_active":"2023-12-17T12:08:49.186419+0000","last_became_peered":"2023-12-17T12:08:49.186419+0000","last_unstale":"2023-12-17T12:09:23.517285+0000","last_undegraded":"2023-12-17T12:09:23.517285+0000","last_fullsized":"2023-12-17T12:09:23.517285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:47:03.035599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511477+0000","last_change":"2023-12-17T12:08:49.192587+0000","last_active":"2023-12-17T12:09:23.511477+0000","last_peered":"2023-12-17T12:09:23.511477+0000","last_clean":"2023-12-17T12:09:23.511477+0000","last_became_active":"2023-12-17T12:08:49.192272+0000","last_became_peered":"2023-12-17T12:08:49.192272+0000","last_unstale":"2023-12-17T12:09:23.511477+0000","last_undegraded":"2023-12-17T12:09:23.511477+0000","last_fullsized":"2023-12-17T12:09:23.511477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:45:30.131065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514841+0000","last_change":"2023-12-17T12:08:49.176906+0000","last_active":"2023-12-17T12:09:23.514841+0000","last_peered":"2023-12-17T12:09:23.514841+0000","last_clean":"2023-12-17T12:09:23.514841+0000","last_became_active":"2023-12-17T12:08:49.176591+0000","last_became_peered":"2023-12-17T12:08:49.176591+0000","last_unstale":"2023-12-17T12:09:23.514841+0000","last_undegraded":"2023-12-17T12:09:23.514841+0000","last_fullsized":"2023-12-17T12:09:23.514841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:27:17.508628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515618+0000","last_change":"2023-12-17T12:08:49.177409+0000","last_active":"2023-12-17T12:09:23.515618+0000","last_peered":"2023-12-17T12:09:23.515618+0000","last_clean":"2023-12-17T12:09:23.515618+0000","last_became_active":"2023-12-17T12:08:49.177093+0000","last_became_peered":"2023-12-17T12:08:49.177093+0000","last_unstale":"2023-12-17T12:09:23.515618+0000","last_undegraded":"2023-12-17T12:09:23.515618+0000","last_fullsized":"2023-12-17T12:09:23.515618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:56:30.669406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519105+0000","last_change":"2023-12-17T12:08:49.194035+0000","last_active":"2023-12-17T12:09:23.519105+0000","last_peered":"2023-12-17T12:09:23.519105+0000","last_clean":"2023-12-17T12:09:23.519105+0000","last_became_active":"2023-12-17T12:08:49.191677+0000","last_became_peered":"2023-12-17T12:08:49.191677+0000","last_unstale":"2023-12-17T12:09:23.519105+0000","last_undegraded":"2023-12-17T12:09:23.519105+0000","last_fullsized":"2023-12-17T12:09:23.519105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:21:00.984977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511867+0000","last_change":"2023-12-17T12:08:49.187140+0000","last_active":"2023-12-17T12:09:23.511867+0000","last_peered":"2023-12-17T12:09:23.511867+0000","last_clean":"2023-12-17T12:09:23.511867+0000","last_became_active":"2023-12-17T12:08:49.186807+0000","last_became_peered":"2023-12-17T12:08:49.186807+0000","last_unstale":"2023-12-17T12:09:23.511867+0000","last_undegraded":"2023-12-17T12:09:23.511867+0000","last_fullsized":"2023-12-17T12:09:23.511867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:28:59.060986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514453+0000","last_change":"2023-12-17T12:08:49.186371+0000","last_active":"2023-12-17T12:09:23.514453+0000","last_peered":"2023-12-17T12:09:23.514453+0000","last_clean":"2023-12-17T12:09:23.514453+0000","last_became_active":"2023-12-17T12:08:49.186045+0000","last_became_peered":"2023-12-17T12:08:49.186045+0000","last_unstale":"2023-12-17T12:09:23.514453+0000","last_undegraded":"2023-12-17T12:09:23.514453+0000","last_fullsized":"2023-12-17T12:09:23.514453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:56:47.851617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517074+0000","last_change":"2023-12-17T12:08:49.179088+0000","last_active":"2023-12-17T12:09:23.517074+0000","last_peered":"2023-12-17T12:09:23.517074+0000","last_clean":"2023-12-17T12:09:23.517074+0000","last_became_active":"2023-12-17T12:08:49.178545+0000","last_became_peered":"2023-12-17T12:08:49.178545+0000","last_unstale":"2023-12-17T12:09:23.517074+0000","last_undegraded":"2023-12-17T12:09:23.517074+0000","last_fullsized":"2023-12-17T12:09:23.517074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:52:58.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517198+0000","last_change":"2023-12-17T12:08:49.187681+0000","last_active":"2023-12-17T12:09:23.517198+0000","last_peered":"2023-12-17T12:09:23.517198+0000","last_clean":"2023-12-17T12:09:23.517198+0000","last_became_active":"2023-12-17T12:08:49.187086+0000","last_became_peered":"2023-12-17T12:08:49.187086+0000","last_unstale":"2023-12-17T12:09:23.517198+0000","last_undegraded":"2023-12-17T12:09:23.517198+0000","last_fullsized":"2023-12-17T12:09:23.517198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:16:55.476924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.513983+0000","last_change":"2023-12-17T12:08:49.202082+0000","last_active":"2023-12-17T12:09:23.513983+0000","last_peered":"2023-12-17T12:09:23.513983+0000","last_clean":"2023-12-17T12:09:23.513983+0000","last_became_active":"2023-12-17T12:08:49.200583+0000","last_became_peered":"2023-12-17T12:08:49.200583+0000","last_unstale":"2023-12-17T12:09:23.513983+0000","last_undegraded":"2023-12-17T12:09:23.513983+0000","last_fullsized":"2023-12-17T12:09:23.513983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:19:25.112916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517610+0000","last_change":"2023-12-17T12:08:49.184442+0000","last_active":"2023-12-17T12:09:23.517610+0000","last_peered":"2023-12-17T12:09:23.517610+0000","last_clean":"2023-12-17T12:09:23.517610+0000","last_became_active":"2023-12-17T12:08:49.184053+0000","last_became_peered":"2023-12-17T12:08:49.184053+0000","last_unstale":"2023-12-17T12:09:23.517610+0000","last_undegraded":"2023-12-17T12:09:23.517610+0000","last_fullsized":"2023-12-17T12:09:23.517610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:38:18.864989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517395+0000","last_change":"2023-12-17T12:08:49.185550+0000","last_active":"2023-12-17T12:09:23.517395+0000","last_peered":"2023-12-17T12:09:23.517395+0000","last_clean":"2023-12-17T12:09:23.517395+0000","last_became_active":"2023-12-17T12:08:49.185190+0000","last_became_peered":"2023-12-17T12:08:49.185190+0000","last_unstale":"2023-12-17T12:09:23.517395+0000","last_undegraded":"2023-12-17T12:09:23.517395+0000","last_fullsized":"2023-12-17T12:09:23.517395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:11:29.821074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.510773+0000","last_change":"2023-12-17T12:08:49.182738+0000","last_active":"2023-12-17T12:09:23.510773+0000","last_peered":"2023-12-17T12:09:23.510773+0000","last_clean":"2023-12-17T12:09:23.510773+0000","last_became_active":"2023-12-17T12:08:49.182451+0000","last_became_peered":"2023-12-17T12:08:49.182451+0000","last_unstale":"2023-12-17T12:09:23.510773+0000","last_undegraded":"2023-12-17T12:09:23.510773+0000","last_fullsized":"2023-12-17T12:09:23.510773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:21:06.788880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515877+0000","last_change":"2023-12-17T12:08:49.182062+0000","last_active":"2023-12-17T12:09:23.515877+0000","last_peered":"2023-12-17T12:09:23.515877+0000","last_clean":"2023-12-17T12:09:23.515877+0000","last_became_active":"2023-12-17T12:08:49.181725+0000","last_became_peered":"2023-12-17T12:08:49.181725+0000","last_unstale":"2023-12-17T12:09:23.515877+0000","last_undegraded":"2023-12-17T12:09:23.515877+0000","last_fullsized":"2023-12-17T12:09:23.515877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:06:26.626520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518680+0000","last_change":"2023-12-17T12:08:49.174849+0000","last_active":"2023-12-17T12:09:23.518680+0000","last_peered":"2023-12-17T12:09:23.518680+0000","last_clean":"2023-12-17T12:09:23.518680+0000","last_became_active":"2023-12-17T12:08:49.174408+0000","last_became_peered":"2023-12-17T12:08:49.174408+0000","last_unstale":"2023-12-17T12:09:23.518680+0000","last_undegraded":"2023-12-17T12:09:23.518680+0000","last_fullsized":"2023-12-17T12:09:23.518680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:36:11.050556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515127+0000","last_change":"2023-12-17T12:08:49.202639+0000","last_active":"2023-12-17T12:09:23.515127+0000","last_peered":"2023-12-17T12:09:23.515127+0000","last_clean":"2023-12-17T12:09:23.515127+0000","last_became_active":"2023-12-17T12:08:49.202400+0000","last_became_peered":"2023-12-17T12:08:49.202400+0000","last_unstale":"2023-12-17T12:09:23.515127+0000","last_undegraded":"2023-12-17T12:09:23.515127+0000","last_fullsized":"2023-12-17T12:09:23.515127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:45:27.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.532321+0000","last_change":"2023-12-17T12:09:24.532321+0000","last_active":"2023-12-17T12:09:24.532321+0000","last_peered":"2023-12-17T12:09:24.532321+0000","last_clean":"2023-12-17T12:09:24.532321+0000","last_became_active":"2023-12-17T12:09:24.521726+0000","last_became_peered":"2023-12-17T12:09:24.521726+0000","last_unstale":"2023-12-17T12:09:24.532321+0000","last_undegraded":"2023-12-17T12:09:24.532321+0000","last_fullsized":"2023-12-17T12:09:24.532321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:28:15.851568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520325+0000","last_change":"2023-12-17T12:08:49.196221+0000","last_active":"2023-12-17T12:09:23.520325+0000","last_peered":"2023-12-17T12:09:23.520325+0000","last_clean":"2023-12-17T12:09:23.520325+0000","last_became_active":"2023-12-17T12:08:49.195908+0000","last_became_peered":"2023-12-17T12:08:49.195908+0000","last_unstale":"2023-12-17T12:09:23.520325+0000","last_undegraded":"2023-12-17T12:09:23.520325+0000","last_fullsized":"2023-12-17T12:09:23.520325+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:28:17.710818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520412+0000","last_change":"2023-12-17T12:08:48.408069+0000","last_active":"2023-12-17T12:09:23.520412+0000","last_peered":"2023-12-17T12:09:23.520412+0000","last_clean":"2023-12-17T12:09:23.520412+0000","last_became_active":"2023-12-17T12:08:48.407209+0000","last_became_peered":"2023-12-17T12:08:48.407209+0000","last_unstale":"2023-12-17T12:09:23.520412+0000","last_undegraded":"2023-12-17T12:09:23.520412+0000","last_fullsized":"2023-12-17T12:09:23.520412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:24:08.191642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519233+0000","last_change":"2023-12-17T12:08:49.179982+0000","last_active":"2023-12-17T12:09:23.519233+0000","last_peered":"2023-12-17T12:09:23.519233+0000","last_clean":"2023-12-17T12:09:23.519233+0000","last_became_active":"2023-12-17T12:08:49.179610+0000","last_became_peered":"2023-12-17T12:08:49.179610+0000","last_unstale":"2023-12-17T12:09:23.519233+0000","last_undegraded":"2023-12-17T12:09:23.519233+0000","last_fullsized":"2023-12-17T12:09:23.519233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:39:13.133693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511984+0000","last_change":"2023-12-17T12:08:48.395609+0000","last_active":"2023-12-17T12:09:23.511984+0000","last_peered":"2023-12-17T12:09:23.511984+0000","last_clean":"2023-12-17T12:09:23.511984+0000","last_became_active":"2023-12-17T12:08:48.393809+0000","last_became_peered":"2023-12-17T12:08:48.393809+0000","last_unstale":"2023-12-17T12:09:23.511984+0000","last_undegraded":"2023-12-17T12:09:23.511984+0000","last_fullsized":"2023-12-17T12:09:23.511984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:45:39.698185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.510850+0000","last_change":"2023-12-17T12:08:49.182636+0000","last_active":"2023-12-17T12:09:23.510850+0000","last_peered":"2023-12-17T12:09:23.510850+0000","last_clean":"2023-12-17T12:09:23.510850+0000","last_became_active":"2023-12-17T12:08:49.182328+0000","last_became_peered":"2023-12-17T12:08:49.182328+0000","last_unstale":"2023-12-17T12:09:23.510850+0000","last_undegraded":"2023-12-17T12:09:23.510850+0000","last_fullsized":"2023-12-17T12:09:23.510850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:19:27.674053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511750+0000","last_change":"2023-12-17T12:08:48.405333+0000","last_active":"2023-12-17T12:09:23.511750+0000","last_peered":"2023-12-17T12:09:23.511750+0000","last_clean":"2023-12-17T12:09:23.511750+0000","last_became_active":"2023-12-17T12:08:48.403558+0000","last_became_peered":"2023-12-17T12:08:48.403558+0000","last_unstale":"2023-12-17T12:09:23.511750+0000","last_undegraded":"2023-12-17T12:09:23.511750+0000","last_fullsized":"2023-12-17T12:09:23.511750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:16:48.657674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517442+0000","last_change":"2023-12-17T12:08:49.187999+0000","last_active":"2023-12-17T12:09:23.517442+0000","last_peered":"2023-12-17T12:09:23.517442+0000","last_clean":"2023-12-17T12:09:23.517442+0000","last_became_active":"2023-12-17T12:08:49.187698+0000","last_became_peered":"2023-12-17T12:08:49.187698+0000","last_unstale":"2023-12-17T12:09:23.517442+0000","last_undegraded":"2023-12-17T12:09:23.517442+0000","last_fullsized":"2023-12-17T12:09:23.517442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:52:22.279227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:32.494769+0000","last_change":"2023-12-17T12:08:48.403205+0000","last_active":"2023-12-17T12:09:32.494769+0000","last_peered":"2023-12-17T12:09:32.494769+0000","last_clean":"2023-12-17T12:09:32.494769+0000","last_became_active":"2023-12-17T12:08:48.401930+0000","last_became_peered":"2023-12-17T12:08:48.401930+0000","last_unstale":"2023-12-17T12:09:32.494769+0000","last_undegraded":"2023-12-17T12:09:32.494769+0000","last_fullsized":"2023-12-17T12:09:32.494769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:28:03.889769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511409+0000","last_change":"2023-12-17T12:08:49.176597+0000","last_active":"2023-12-17T12:09:23.511409+0000","last_peered":"2023-12-17T12:09:23.511409+0000","last_clean":"2023-12-17T12:09:23.511409+0000","last_became_active":"2023-12-17T12:08:49.176279+0000","last_became_peered":"2023-12-17T12:08:49.176279+0000","last_unstale":"2023-12-17T12:09:23.511409+0000","last_undegraded":"2023-12-17T12:09:23.511409+0000","last_fullsized":"2023-12-17T12:09:23.511409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:38:48.558555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514043+0000","last_change":"2023-12-17T12:08:48.390768+0000","last_active":"2023-12-17T12:09:23.514043+0000","last_peered":"2023-12-17T12:09:23.514043+0000","last_clean":"2023-12-17T12:09:23.514043+0000","last_became_active":"2023-12-17T12:08:48.390452+0000","last_became_peered":"2023-12-17T12:08:48.390452+0000","last_unstale":"2023-12-17T12:09:23.514043+0000","last_undegraded":"2023-12-17T12:09:23.514043+0000","last_fullsized":"2023-12-17T12:09:23.514043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:16:50.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516911+0000","last_change":"2023-12-17T12:08:49.194554+0000","last_active":"2023-12-17T12:09:23.516911+0000","last_peered":"2023-12-17T12:09:23.516911+0000","last_clean":"2023-12-17T12:09:23.516911+0000","last_became_active":"2023-12-17T12:08:49.193794+0000","last_became_peered":"2023-12-17T12:08:49.193794+0000","last_unstale":"2023-12-17T12:09:23.516911+0000","last_undegraded":"2023-12-17T12:09:23.516911+0000","last_fullsized":"2023-12-17T12:09:23.516911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:19:30.140621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511983+0000","last_change":"2023-12-17T12:08:48.388949+0000","last_active":"2023-12-17T12:09:23.511983+0000","last_peered":"2023-12-17T12:09:23.511983+0000","last_clean":"2023-12-17T12:09:23.511983+0000","last_became_active":"2023-12-17T12:08:48.386904+0000","last_became_peered":"2023-12-17T12:08:48.386904+0000","last_unstale":"2023-12-17T12:09:23.511983+0000","last_undegraded":"2023-12-17T12:09:23.511983+0000","last_fullsized":"2023-12-17T12:09:23.511983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:25:42.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519088+0000","last_change":"2023-12-17T12:08:49.197069+0000","last_active":"2023-12-17T12:09:23.519088+0000","last_peered":"2023-12-17T12:09:23.519088+0000","last_clean":"2023-12-17T12:09:23.519088+0000","last_became_active":"2023-12-17T12:08:49.196366+0000","last_became_peered":"2023-12-17T12:08:49.196366+0000","last_unstale":"2023-12-17T12:09:23.519088+0000","last_undegraded":"2023-12-17T12:09:23.519088+0000","last_fullsized":"2023-12-17T12:09:23.519088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:43:41.262255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.522956+0000","last_change":"2023-12-17T12:09:24.522454+0000","last_active":"2023-12-17T12:09:24.522956+0000","last_peered":"2023-12-17T12:09:24.522956+0000","last_clean":"2023-12-17T12:09:24.522956+0000","last_became_active":"2023-12-17T12:09:24.521938+0000","last_became_peered":"2023-12-17T12:09:24.521938+0000","last_unstale":"2023-12-17T12:09:24.522956+0000","last_undegraded":"2023-12-17T12:09:24.522956+0000","last_fullsized":"2023-12-17T12:09:24.522956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:59:54.789618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515897+0000","last_change":"2023-12-17T12:08:49.192837+0000","last_active":"2023-12-17T12:09:23.515897+0000","last_peered":"2023-12-17T12:09:23.515897+0000","last_clean":"2023-12-17T12:09:23.515897+0000","last_became_active":"2023-12-17T12:08:49.192479+0000","last_became_peered":"2023-12-17T12:08:49.192479+0000","last_unstale":"2023-12-17T12:09:23.515897+0000","last_undegraded":"2023-12-17T12:09:23.515897+0000","last_fullsized":"2023-12-17T12:09:23.515897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:08:50.771280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":57,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:32.493884+0000","last_change":"2023-12-17T12:08:48.395661+0000","last_active":"2023-12-17T12:09:32.493884+0000","last_peered":"2023-12-17T12:09:32.493884+0000","last_clean":"2023-12-17T12:09:32.493884+0000","last_became_active":"2023-12-17T12:08:48.395282+0000","last_became_peered":"2023-12-17T12:08:48.395282+0000","last_unstale":"2023-12-17T12:09:32.493884+0000","last_undegraded":"2023-12-17T12:09:32.493884+0000","last_fullsized":"2023-12-17T12:09:32.493884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:02:15.554520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511299+0000","last_change":"2023-12-17T12:08:49.179339+0000","last_active":"2023-12-17T12:09:23.511299+0000","last_peered":"2023-12-17T12:09:23.511299+0000","last_clean":"2023-12-17T12:09:23.511299+0000","last_became_active":"2023-12-17T12:08:49.179041+0000","last_became_peered":"2023-12-17T12:08:49.179041+0000","last_unstale":"2023-12-17T12:09:23.511299+0000","last_undegraded":"2023-12-17T12:09:23.511299+0000","last_fullsized":"2023-12-17T12:09:23.511299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:27:28.508576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517241+0000","last_change":"2023-12-17T12:08:48.401911+0000","last_active":"2023-12-17T12:09:23.517241+0000","last_peered":"2023-12-17T12:09:23.517241+0000","last_clean":"2023-12-17T12:09:23.517241+0000","last_became_active":"2023-12-17T12:08:48.401552+0000","last_became_peered":"2023-12-17T12:08:48.401552+0000","last_unstale":"2023-12-17T12:09:23.517241+0000","last_undegraded":"2023-12-17T12:09:23.517241+0000","last_fullsized":"2023-12-17T12:09:23.517241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:40:54.813575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519095+0000","last_change":"2023-12-17T12:08:49.177683+0000","last_active":"2023-12-17T12:09:23.519095+0000","last_peered":"2023-12-17T12:09:23.519095+0000","last_clean":"2023-12-17T12:09:23.519095+0000","last_became_active":"2023-12-17T12:08:49.177313+0000","last_became_peered":"2023-12-17T12:08:49.177313+0000","last_unstale":"2023-12-17T12:09:23.519095+0000","last_undegraded":"2023-12-17T12:09:23.519095+0000","last_fullsized":"2023-12-17T12:09:23.519095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:29:40.397954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517631+0000","last_change":"2023-12-17T12:08:48.392907+0000","last_active":"2023-12-17T12:09:23.517631+0000","last_peered":"2023-12-17T12:09:23.517631+0000","last_clean":"2023-12-17T12:09:23.517631+0000","last_became_active":"2023-12-17T12:08:48.392504+0000","last_became_peered":"2023-12-17T12:08:48.392504+0000","last_unstale":"2023-12-17T12:09:23.517631+0000","last_undegraded":"2023-12-17T12:09:23.517631+0000","last_fullsized":"2023-12-17T12:09:23.517631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:25:24.561271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511030+0000","last_change":"2023-12-17T12:08:49.186687+0000","last_active":"2023-12-17T12:09:23.511030+0000","last_peered":"2023-12-17T12:09:23.511030+0000","last_clean":"2023-12-17T12:09:23.511030+0000","last_became_active":"2023-12-17T12:08:49.186400+0000","last_became_peered":"2023-12-17T12:08:49.186400+0000","last_unstale":"2023-12-17T12:09:23.511030+0000","last_undegraded":"2023-12-17T12:09:23.511030+0000","last_fullsized":"2023-12-17T12:09:23.511030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:12:27.143793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'25","reported_seq":145,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:34.072478+0000","last_change":"2023-12-17T12:09:25.063100+0000","last_active":"2023-12-17T12:09:34.072478+0000","last_peered":"2023-12-17T12:09:34.072478+0000","last_clean":"2023-12-17T12:09:34.072478+0000","last_became_active":"2023-12-17T12:09:25.051399+0000","last_became_peered":"2023-12-17T12:09:25.051399+0000","last_unstale":"2023-12-17T12:09:34.072478+0000","last_undegraded":"2023-12-17T12:09:34.072478+0000","last_fullsized":"2023-12-17T12:09:34.072478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:17:57.842293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57804,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":77,"num_read_kb":94,"num_write":26,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57804,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511762+0000","last_change":"2023-12-17T12:08:49.176410+0000","last_active":"2023-12-17T12:09:23.511762+0000","last_peered":"2023-12-17T12:09:23.511762+0000","last_clean":"2023-12-17T12:09:23.511762+0000","last_became_active":"2023-12-17T12:08:49.176120+0000","last_became_peered":"2023-12-17T12:08:49.176120+0000","last_unstale":"2023-12-17T12:09:23.511762+0000","last_undegraded":"2023-12-17T12:09:23.511762+0000","last_fullsized":"2023-12-17T12:09:23.511762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:28:18.241319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514401+0000","last_change":"2023-12-17T12:08:48.405781+0000","last_active":"2023-12-17T12:09:23.514401+0000","last_peered":"2023-12-17T12:09:23.514401+0000","last_clean":"2023-12-17T12:09:23.514401+0000","last_became_active":"2023-12-17T12:08:48.405255+0000","last_became_peered":"2023-12-17T12:08:48.405255+0000","last_unstale":"2023-12-17T12:09:23.514401+0000","last_undegraded":"2023-12-17T12:09:23.514401+0000","last_fullsized":"2023-12-17T12:09:23.514401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:00:10.046901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511949+0000","last_change":"2023-12-17T12:08:49.199457+0000","last_active":"2023-12-17T12:09:23.511949+0000","last_peered":"2023-12-17T12:09:23.511949+0000","last_clean":"2023-12-17T12:09:23.511949+0000","last_became_active":"2023-12-17T12:08:49.197147+0000","last_became_peered":"2023-12-17T12:08:49.197147+0000","last_unstale":"2023-12-17T12:09:23.511949+0000","last_undegraded":"2023-12-17T12:09:23.511949+0000","last_fullsized":"2023-12-17T12:09:23.511949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:36:46.619674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517016+0000","last_change":"2023-12-17T12:08:48.407127+0000","last_active":"2023-12-17T12:09:23.517016+0000","last_peered":"2023-12-17T12:09:23.517016+0000","last_clean":"2023-12-17T12:09:23.517016+0000","last_became_active":"2023-12-17T12:08:48.406726+0000","last_became_peered":"2023-12-17T12:08:48.406726+0000","last_unstale":"2023-12-17T12:09:23.517016+0000","last_undegraded":"2023-12-17T12:09:23.517016+0000","last_fullsized":"2023-12-17T12:09:23.517016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:16:19.492428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.523348+0000","last_change":"2023-12-17T12:09:24.523348+0000","last_active":"2023-12-17T12:09:24.523348+0000","last_peered":"2023-12-17T12:09:24.523348+0000","last_clean":"2023-12-17T12:09:24.523348+0000","last_became_active":"2023-12-17T12:09:24.523000+0000","last_became_peered":"2023-12-17T12:09:24.523000+0000","last_unstale":"2023-12-17T12:09:24.523348+0000","last_undegraded":"2023-12-17T12:09:24.523348+0000","last_fullsized":"2023-12-17T12:09:24.523348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:22:21.503203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:34.497607+0000","last_change":"2023-12-17T12:08:48.393092+0000","last_active":"2023-12-17T12:09:34.497607+0000","last_peered":"2023-12-17T12:09:34.497607+0000","last_clean":"2023-12-17T12:09:34.497607+0000","last_became_active":"2023-12-17T12:08:48.392702+0000","last_became_peered":"2023-12-17T12:08:48.392702+0000","last_unstale":"2023-12-17T12:09:34.497607+0000","last_undegraded":"2023-12-17T12:09:34.497607+0000","last_fullsized":"2023-12-17T12:09:34.497607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:53:00.261870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":977,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516437+0000","last_change":"2023-12-17T12:08:49.181367+0000","last_active":"2023-12-17T12:09:23.516437+0000","last_peered":"2023-12-17T12:09:23.516437+0000","last_clean":"2023-12-17T12:09:23.516437+0000","last_became_active":"2023-12-17T12:08:49.180841+0000","last_became_peered":"2023-12-17T12:08:49.180841+0000","last_unstale":"2023-12-17T12:09:23.516437+0000","last_undegraded":"2023-12-17T12:09:23.516437+0000","last_fullsized":"2023-12-17T12:09:23.516437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:57:33.920366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511292+0000","last_change":"2023-12-17T12:08:48.398863+0000","last_active":"2023-12-17T12:09:23.511292+0000","last_peered":"2023-12-17T12:09:23.511292+0000","last_clean":"2023-12-17T12:09:23.511292+0000","last_became_active":"2023-12-17T12:08:48.396178+0000","last_became_peered":"2023-12-17T12:08:48.396178+0000","last_unstale":"2023-12-17T12:09:23.511292+0000","last_undegraded":"2023-12-17T12:09:23.511292+0000","last_fullsized":"2023-12-17T12:09:23.511292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:31:44.593246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516840+0000","last_change":"2023-12-17T12:08:49.190795+0000","last_active":"2023-12-17T12:09:23.516840+0000","last_peered":"2023-12-17T12:09:23.516840+0000","last_clean":"2023-12-17T12:09:23.516840+0000","last_became_active":"2023-12-17T12:08:49.190468+0000","last_became_peered":"2023-12-17T12:08:49.190468+0000","last_unstale":"2023-12-17T12:09:23.516840+0000","last_undegraded":"2023-12-17T12:09:23.516840+0000","last_fullsized":"2023-12-17T12:09:23.516840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:26:37.302852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.512001+0000","last_change":"2023-12-17T12:08:48.391524+0000","last_active":"2023-12-17T12:09:23.512001+0000","last_peered":"2023-12-17T12:09:23.512001+0000","last_clean":"2023-12-17T12:09:23.512001+0000","last_became_active":"2023-12-17T12:08:48.391195+0000","last_became_peered":"2023-12-17T12:08:48.391195+0000","last_unstale":"2023-12-17T12:09:23.512001+0000","last_undegraded":"2023-12-17T12:09:23.512001+0000","last_fullsized":"2023-12-17T12:09:23.512001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:33:55.399168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515717+0000","last_change":"2023-12-17T12:08:49.188376+0000","last_active":"2023-12-17T12:09:23.515717+0000","last_peered":"2023-12-17T12:09:23.515717+0000","last_clean":"2023-12-17T12:09:23.515717+0000","last_became_active":"2023-12-17T12:08:49.188038+0000","last_became_peered":"2023-12-17T12:08:49.188038+0000","last_unstale":"2023-12-17T12:09:23.515717+0000","last_undegraded":"2023-12-17T12:09:23.515717+0000","last_fullsized":"2023-12-17T12:09:23.515717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:27:35.263053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515192+0000","last_change":"2023-12-17T12:08:48.401451+0000","last_active":"2023-12-17T12:09:23.515192+0000","last_peered":"2023-12-17T12:09:23.515192+0000","last_clean":"2023-12-17T12:09:23.515192+0000","last_became_active":"2023-12-17T12:08:48.401100+0000","last_became_peered":"2023-12-17T12:08:48.401100+0000","last_unstale":"2023-12-17T12:09:23.515192+0000","last_undegraded":"2023-12-17T12:09:23.515192+0000","last_fullsized":"2023-12-17T12:09:23.515192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:38:48.372997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514180+0000","last_change":"2023-12-17T12:08:49.194033+0000","last_active":"2023-12-17T12:09:23.514180+0000","last_peered":"2023-12-17T12:09:23.514180+0000","last_clean":"2023-12-17T12:09:23.514180+0000","last_became_active":"2023-12-17T12:08:49.192344+0000","last_became_peered":"2023-12-17T12:08:49.192344+0000","last_unstale":"2023-12-17T12:09:23.514180+0000","last_undegraded":"2023-12-17T12:09:23.514180+0000","last_fullsized":"2023-12-17T12:09:23.514180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:00:40.106655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515945+0000","last_change":"2023-12-17T12:08:48.397060+0000","last_active":"2023-12-17T12:09:23.515945+0000","last_peered":"2023-12-17T12:09:23.515945+0000","last_clean":"2023-12-17T12:09:23.515945+0000","last_became_active":"2023-12-17T12:08:48.396704+0000","last_became_peered":"2023-12-17T12:08:48.396704+0000","last_unstale":"2023-12-17T12:09:23.515945+0000","last_undegraded":"2023-12-17T12:09:23.515945+0000","last_fullsized":"2023-12-17T12:09:23.515945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:31:08.882089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514200+0000","last_change":"2023-12-17T12:08:49.183915+0000","last_active":"2023-12-17T12:09:23.514200+0000","last_peered":"2023-12-17T12:09:23.514200+0000","last_clean":"2023-12-17T12:09:23.514200+0000","last_became_active":"2023-12-17T12:08:49.183615+0000","last_became_peered":"2023-12-17T12:08:49.183615+0000","last_unstale":"2023-12-17T12:09:23.514200+0000","last_undegraded":"2023-12-17T12:09:23.514200+0000","last_fullsized":"2023-12-17T12:09:23.514200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:29:09.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517302+0000","last_change":"2023-12-17T12:08:49.190952+0000","last_active":"2023-12-17T12:09:23.517302+0000","last_peered":"2023-12-17T12:09:23.517302+0000","last_clean":"2023-12-17T12:09:23.517302+0000","last_became_active":"2023-12-17T12:08:49.190586+0000","last_became_peered":"2023-12-17T12:08:49.190586+0000","last_unstale":"2023-12-17T12:09:23.517302+0000","last_undegraded":"2023-12-17T12:09:23.517302+0000","last_fullsized":"2023-12-17T12:09:23.517302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:44:00.972429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:34.497174+0000","last_change":"2023-12-17T12:08:48.409488+0000","last_active":"2023-12-17T12:09:34.497174+0000","last_peered":"2023-12-17T12:09:34.497174+0000","last_clean":"2023-12-17T12:09:34.497174+0000","last_became_active":"2023-12-17T12:08:48.408703+0000","last_became_peered":"2023-12-17T12:08:48.408703+0000","last_unstale":"2023-12-17T12:09:34.497174+0000","last_undegraded":"2023-12-17T12:09:34.497174+0000","last_fullsized":"2023-12-17T12:09:34.497174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:33:14.861530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519847+0000","last_change":"2023-12-17T12:08:49.197960+0000","last_active":"2023-12-17T12:09:23.519847+0000","last_peered":"2023-12-17T12:09:23.519847+0000","last_clean":"2023-12-17T12:09:23.519847+0000","last_became_active":"2023-12-17T12:08:49.196484+0000","last_became_peered":"2023-12-17T12:08:49.196484+0000","last_unstale":"2023-12-17T12:09:23.519847+0000","last_undegraded":"2023-12-17T12:09:23.519847+0000","last_fullsized":"2023-12-17T12:09:23.519847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:53:50.293026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518375+0000","last_change":"2023-12-17T12:08:48.387589+0000","last_active":"2023-12-17T12:09:23.518375+0000","last_peered":"2023-12-17T12:09:23.518375+0000","last_clean":"2023-12-17T12:09:23.518375+0000","last_became_active":"2023-12-17T12:08:48.387258+0000","last_became_peered":"2023-12-17T12:08:48.387258+0000","last_unstale":"2023-12-17T12:09:23.518375+0000","last_undegraded":"2023-12-17T12:09:23.518375+0000","last_fullsized":"2023-12-17T12:09:23.518375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:29:30.815005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516549+0000","last_change":"2023-12-17T12:07:42.894663+0000","last_active":"2023-12-17T12:09:23.516549+0000","last_peered":"2023-12-17T12:09:23.516549+0000","last_clean":"2023-12-17T12:09:23.516549+0000","last_became_active":"2023-12-17T12:07:42.894199+0000","last_became_peered":"2023-12-17T12:07:42.894199+0000","last_unstale":"2023-12-17T12:09:23.516549+0000","last_undegraded":"2023-12-17T12:09:23.516549+0000","last_fullsized":"2023-12-17T12:09:23.516549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:04:41.531220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:04:41.531220+0000","last_clean_scrub_stamp":"2023-12-17T12:04:41.531220+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:07:49.358197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511873+0000","last_change":"2023-12-17T12:08:49.195358+0000","last_active":"2023-12-17T12:09:23.511873+0000","last_peered":"2023-12-17T12:09:23.511873+0000","last_clean":"2023-12-17T12:09:23.511873+0000","last_became_active":"2023-12-17T12:08:49.194996+0000","last_became_peered":"2023-12-17T12:08:49.194996+0000","last_unstale":"2023-12-17T12:09:23.511873+0000","last_undegraded":"2023-12-17T12:09:23.511873+0000","last_fullsized":"2023-12-17T12:09:23.511873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:23:10.016755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516724+0000","last_change":"2023-12-17T12:08:48.394937+0000","last_active":"2023-12-17T12:09:23.516724+0000","last_peered":"2023-12-17T12:09:23.516724+0000","last_clean":"2023-12-17T12:09:23.516724+0000","last_became_active":"2023-12-17T12:08:48.394590+0000","last_became_peered":"2023-12-17T12:08:48.394590+0000","last_unstale":"2023-12-17T12:09:23.516724+0000","last_undegraded":"2023-12-17T12:09:23.516724+0000","last_fullsized":"2023-12-17T12:09:23.516724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:24:49.088883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.512088+0000","last_change":"2023-12-17T12:08:48.396769+0000","last_active":"2023-12-17T12:09:23.512088+0000","last_peered":"2023-12-17T12:09:23.512088+0000","last_clean":"2023-12-17T12:09:23.512088+0000","last_became_active":"2023-12-17T12:08:48.395984+0000","last_became_peered":"2023-12-17T12:08:48.395984+0000","last_unstale":"2023-12-17T12:09:23.512088+0000","last_undegraded":"2023-12-17T12:09:23.512088+0000","last_fullsized":"2023-12-17T12:09:23.512088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:34:59.872771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.512311+0000","last_change":"2023-12-17T12:08:49.194598+0000","last_active":"2023-12-17T12:09:23.512311+0000","last_peered":"2023-12-17T12:09:23.512311+0000","last_clean":"2023-12-17T12:09:23.512311+0000","last_became_active":"2023-12-17T12:08:49.189450+0000","last_became_peered":"2023-12-17T12:08:49.189450+0000","last_unstale":"2023-12-17T12:09:23.512311+0000","last_undegraded":"2023-12-17T12:09:23.512311+0000","last_fullsized":"2023-12-17T12:09:23.512311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:45:57.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.535949+0000","last_change":"2023-12-17T12:09:24.535949+0000","last_active":"2023-12-17T12:09:24.535949+0000","last_peered":"2023-12-17T12:09:24.535949+0000","last_clean":"2023-12-17T12:09:24.535949+0000","last_became_active":"2023-12-17T12:09:24.526345+0000","last_became_peered":"2023-12-17T12:09:24.526345+0000","last_unstale":"2023-12-17T12:09:24.535949+0000","last_undegraded":"2023-12-17T12:09:24.535949+0000","last_fullsized":"2023-12-17T12:09:24.535949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:50:30.756017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520562+0000","last_change":"2023-12-17T12:08:49.196489+0000","last_active":"2023-12-17T12:09:23.520562+0000","last_peered":"2023-12-17T12:09:23.520562+0000","last_clean":"2023-12-17T12:09:23.520562+0000","last_became_active":"2023-12-17T12:08:49.196063+0000","last_became_peered":"2023-12-17T12:08:49.196063+0000","last_unstale":"2023-12-17T12:09:23.520562+0000","last_undegraded":"2023-12-17T12:09:23.520562+0000","last_fullsized":"2023-12-17T12:09:23.520562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:16:08.658987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518017+0000","last_change":"2023-12-17T12:08:48.400075+0000","last_active":"2023-12-17T12:09:23.518017+0000","last_peered":"2023-12-17T12:09:23.518017+0000","last_clean":"2023-12-17T12:09:23.518017+0000","last_became_active":"2023-12-17T12:08:48.398728+0000","last_became_peered":"2023-12-17T12:08:48.398728+0000","last_unstale":"2023-12-17T12:09:23.518017+0000","last_undegraded":"2023-12-17T12:09:23.518017+0000","last_fullsized":"2023-12-17T12:09:23.518017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:10:21.065038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518876+0000","last_change":"2023-12-17T12:08:49.193200+0000","last_active":"2023-12-17T12:09:23.518876+0000","last_peered":"2023-12-17T12:09:23.518876+0000","last_clean":"2023-12-17T12:09:23.518876+0000","last_became_active":"2023-12-17T12:08:49.192855+0000","last_became_peered":"2023-12-17T12:08:49.192855+0000","last_unstale":"2023-12-17T12:09:23.518876+0000","last_undegraded":"2023-12-17T12:09:23.518876+0000","last_fullsized":"2023-12-17T12:09:23.518876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:14:09.089956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.510772+0000","last_change":"2023-12-17T12:08:48.404661+0000","last_active":"2023-12-17T12:09:23.510772+0000","last_peered":"2023-12-17T12:09:23.510772+0000","last_clean":"2023-12-17T12:09:23.510772+0000","last_became_active":"2023-12-17T12:08:48.404336+0000","last_became_peered":"2023-12-17T12:08:48.404336+0000","last_unstale":"2023-12-17T12:09:23.510772+0000","last_undegraded":"2023-12-17T12:09:23.510772+0000","last_fullsized":"2023-12-17T12:09:23.510772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:37:20.863966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511268+0000","last_change":"2023-12-17T12:08:49.181169+0000","last_active":"2023-12-17T12:09:23.511268+0000","last_peered":"2023-12-17T12:09:23.511268+0000","last_clean":"2023-12-17T12:09:23.511268+0000","last_became_active":"2023-12-17T12:08:49.177802+0000","last_became_peered":"2023-12-17T12:08:49.177802+0000","last_unstale":"2023-12-17T12:09:23.511268+0000","last_undegraded":"2023-12-17T12:09:23.511268+0000","last_fullsized":"2023-12-17T12:09:23.511268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:56:50.122246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519361+0000","last_change":"2023-12-17T12:08:48.407865+0000","last_active":"2023-12-17T12:09:23.519361+0000","last_peered":"2023-12-17T12:09:23.519361+0000","last_clean":"2023-12-17T12:09:23.519361+0000","last_became_active":"2023-12-17T12:08:48.407580+0000","last_became_peered":"2023-12-17T12:08:48.407580+0000","last_unstale":"2023-12-17T12:09:23.519361+0000","last_undegraded":"2023-12-17T12:09:23.519361+0000","last_fullsized":"2023-12-17T12:09:23.519361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:10:20.505630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515970+0000","last_change":"2023-12-17T12:08:49.192724+0000","last_active":"2023-12-17T12:09:23.515970+0000","last_peered":"2023-12-17T12:09:23.515970+0000","last_clean":"2023-12-17T12:09:23.515970+0000","last_became_active":"2023-12-17T12:08:49.188658+0000","last_became_peered":"2023-12-17T12:08:49.188658+0000","last_unstale":"2023-12-17T12:09:23.515970+0000","last_undegraded":"2023-12-17T12:09:23.515970+0000","last_fullsized":"2023-12-17T12:09:23.515970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:00:17.493003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514328+0000","last_change":"2023-12-17T12:08:48.404717+0000","last_active":"2023-12-17T12:09:23.514328+0000","last_peered":"2023-12-17T12:09:23.514328+0000","last_clean":"2023-12-17T12:09:23.514328+0000","last_became_active":"2023-12-17T12:08:48.404426+0000","last_became_peered":"2023-12-17T12:08:48.404426+0000","last_unstale":"2023-12-17T12:09:23.514328+0000","last_undegraded":"2023-12-17T12:09:23.514328+0000","last_fullsized":"2023-12-17T12:09:23.514328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:35:28.090855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520139+0000","last_change":"2023-12-17T12:08:49.191911+0000","last_active":"2023-12-17T12:09:23.520139+0000","last_peered":"2023-12-17T12:09:23.520139+0000","last_clean":"2023-12-17T12:09:23.520139+0000","last_became_active":"2023-12-17T12:08:49.191573+0000","last_became_peered":"2023-12-17T12:08:49.191573+0000","last_unstale":"2023-12-17T12:09:23.520139+0000","last_undegraded":"2023-12-17T12:09:23.520139+0000","last_fullsized":"2023-12-17T12:09:23.520139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:39:35.354221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:25.056633+0000","last_change":"2023-12-17T12:09:25.056633+0000","last_active":"2023-12-17T12:09:25.056633+0000","last_peered":"2023-12-17T12:09:25.056633+0000","last_clean":"2023-12-17T12:09:25.056633+0000","last_became_active":"2023-12-17T12:09:25.051396+0000","last_became_peered":"2023-12-17T12:09:25.051396+0000","last_unstale":"2023-12-17T12:09:25.056633+0000","last_undegraded":"2023-12-17T12:09:25.056633+0000","last_fullsized":"2023-12-17T12:09:25.056633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:50:14.091049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516058+0000","last_change":"2023-12-17T12:08:49.191906+0000","last_active":"2023-12-17T12:09:23.516058+0000","last_peered":"2023-12-17T12:09:23.516058+0000","last_clean":"2023-12-17T12:09:23.516058+0000","last_became_active":"2023-12-17T12:08:49.191572+0000","last_became_peered":"2023-12-17T12:08:49.191572+0000","last_unstale":"2023-12-17T12:09:23.516058+0000","last_undegraded":"2023-12-17T12:09:23.516058+0000","last_fullsized":"2023-12-17T12:09:23.516058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:56:05.670034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515249+0000","last_change":"2023-12-17T12:08:48.399004+0000","last_active":"2023-12-17T12:09:23.515249+0000","last_peered":"2023-12-17T12:09:23.515249+0000","last_clean":"2023-12-17T12:09:23.515249+0000","last_became_active":"2023-12-17T12:08:48.398453+0000","last_became_peered":"2023-12-17T12:08:48.398453+0000","last_unstale":"2023-12-17T12:09:23.515249+0000","last_undegraded":"2023-12-17T12:09:23.515249+0000","last_fullsized":"2023-12-17T12:09:23.515249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:30:53.767008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517280+0000","last_change":"2023-12-17T12:08:49.178859+0000","last_active":"2023-12-17T12:09:23.517280+0000","last_peered":"2023-12-17T12:09:23.517280+0000","last_clean":"2023-12-17T12:09:23.517280+0000","last_became_active":"2023-12-17T12:08:49.178119+0000","last_became_peered":"2023-12-17T12:08:49.178119+0000","last_unstale":"2023-12-17T12:09:23.517280+0000","last_undegraded":"2023-12-17T12:09:23.517280+0000","last_fullsized":"2023-12-17T12:09:23.517280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:32:40.758242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517279+0000","last_change":"2023-12-17T12:08:48.407249+0000","last_active":"2023-12-17T12:09:23.517279+0000","last_peered":"2023-12-17T12:09:23.517279+0000","last_clean":"2023-12-17T12:09:23.517279+0000","last_became_active":"2023-12-17T12:08:48.406572+0000","last_became_peered":"2023-12-17T12:08:48.406572+0000","last_unstale":"2023-12-17T12:09:23.517279+0000","last_undegraded":"2023-12-17T12:09:23.517279+0000","last_fullsized":"2023-12-17T12:09:23.517279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:19:07.751233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.527971+0000","last_change":"2023-12-17T12:09:24.527971+0000","last_active":"2023-12-17T12:09:24.527971+0000","last_peered":"2023-12-17T12:09:24.527971+0000","last_clean":"2023-12-17T12:09:24.527971+0000","last_became_active":"2023-12-17T12:09:24.523308+0000","last_became_peered":"2023-12-17T12:09:24.523308+0000","last_unstale":"2023-12-17T12:09:24.527971+0000","last_undegraded":"2023-12-17T12:09:24.527971+0000","last_fullsized":"2023-12-17T12:09:24.527971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:17:08.560065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.512717+0000","last_change":"2023-12-17T12:08:48.406799+0000","last_active":"2023-12-17T12:09:23.512717+0000","last_peered":"2023-12-17T12:09:23.512717+0000","last_clean":"2023-12-17T12:09:23.512717+0000","last_became_active":"2023-12-17T12:08:48.406423+0000","last_became_peered":"2023-12-17T12:08:48.406423+0000","last_unstale":"2023-12-17T12:09:23.512717+0000","last_undegraded":"2023-12-17T12:09:23.512717+0000","last_fullsized":"2023-12-17T12:09:23.512717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:43:55.600399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517454+0000","last_change":"2023-12-17T12:08:49.182840+0000","last_active":"2023-12-17T12:09:23.517454+0000","last_peered":"2023-12-17T12:09:23.517454+0000","last_clean":"2023-12-17T12:09:23.517454+0000","last_became_active":"2023-12-17T12:08:49.181562+0000","last_became_peered":"2023-12-17T12:08:49.181562+0000","last_unstale":"2023-12-17T12:09:23.517454+0000","last_undegraded":"2023-12-17T12:09:23.517454+0000","last_fullsized":"2023-12-17T12:09:23.517454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:01:41.059044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514649+0000","last_change":"2023-12-17T12:08:49.193449+0000","last_active":"2023-12-17T12:09:23.514649+0000","last_peered":"2023-12-17T12:09:23.514649+0000","last_clean":"2023-12-17T12:09:23.514649+0000","last_became_active":"2023-12-17T12:08:49.192942+0000","last_became_peered":"2023-12-17T12:08:49.192942+0000","last_unstale":"2023-12-17T12:09:23.514649+0000","last_undegraded":"2023-12-17T12:09:23.514649+0000","last_fullsized":"2023-12-17T12:09:23.514649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:42:51.021808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514217+0000","last_change":"2023-12-17T12:08:49.193515+0000","last_active":"2023-12-17T12:09:23.514217+0000","last_peered":"2023-12-17T12:09:23.514217+0000","last_clean":"2023-12-17T12:09:23.514217+0000","last_became_active":"2023-12-17T12:08:49.193232+0000","last_became_peered":"2023-12-17T12:08:49.193232+0000","last_unstale":"2023-12-17T12:09:23.514217+0000","last_undegraded":"2023-12-17T12:09:23.514217+0000","last_fullsized":"2023-12-17T12:09:23.514217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:26:59.948350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514611+0000","last_change":"2023-12-17T12:08:49.197110+0000","last_active":"2023-12-17T12:09:23.514611+0000","last_peered":"2023-12-17T12:09:23.514611+0000","last_clean":"2023-12-17T12:09:23.514611+0000","last_became_active":"2023-12-17T12:08:49.196712+0000","last_became_peered":"2023-12-17T12:08:49.196712+0000","last_unstale":"2023-12-17T12:09:23.514611+0000","last_undegraded":"2023-12-17T12:09:23.514611+0000","last_fullsized":"2023-12-17T12:09:23.514611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:21:40.555247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517792+0000","last_change":"2023-12-17T12:08:49.192021+0000","last_active":"2023-12-17T12:09:23.517792+0000","last_peered":"2023-12-17T12:09:23.517792+0000","last_clean":"2023-12-17T12:09:23.517792+0000","last_became_active":"2023-12-17T12:08:49.191696+0000","last_became_peered":"2023-12-17T12:08:49.191696+0000","last_unstale":"2023-12-17T12:09:23.517792+0000","last_undegraded":"2023-12-17T12:09:23.517792+0000","last_fullsized":"2023-12-17T12:09:23.517792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:27:40.425617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:25.050811+0000","last_change":"2023-12-17T12:09:25.050811+0000","last_active":"2023-12-17T12:09:25.050811+0000","last_peered":"2023-12-17T12:09:25.050811+0000","last_clean":"2023-12-17T12:09:25.050811+0000","last_became_active":"2023-12-17T12:09:25.050385+0000","last_became_peered":"2023-12-17T12:09:25.050385+0000","last_unstale":"2023-12-17T12:09:25.050811+0000","last_undegraded":"2023-12-17T12:09:25.050811+0000","last_fullsized":"2023-12-17T12:09:25.050811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:43:43.241960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515723+0000","last_change":"2023-12-17T12:08:49.191147+0000","last_active":"2023-12-17T12:09:23.515723+0000","last_peered":"2023-12-17T12:09:23.515723+0000","last_clean":"2023-12-17T12:09:23.515723+0000","last_became_active":"2023-12-17T12:08:49.190804+0000","last_became_peered":"2023-12-17T12:08:49.190804+0000","last_unstale":"2023-12-17T12:09:23.515723+0000","last_undegraded":"2023-12-17T12:09:23.515723+0000","last_fullsized":"2023-12-17T12:09:23.515723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:09:40.551342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515591+0000","last_change":"2023-12-17T12:08:49.184707+0000","last_active":"2023-12-17T12:09:23.515591+0000","last_peered":"2023-12-17T12:09:23.515591+0000","last_clean":"2023-12-17T12:09:23.515591+0000","last_became_active":"2023-12-17T12:08:49.183644+0000","last_became_peered":"2023-12-17T12:08:49.183644+0000","last_unstale":"2023-12-17T12:09:23.515591+0000","last_undegraded":"2023-12-17T12:09:23.515591+0000","last_fullsized":"2023-12-17T12:09:23.515591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:30:43.845054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520483+0000","last_change":"2023-12-17T12:08:49.195299+0000","last_active":"2023-12-17T12:09:23.520483+0000","last_peered":"2023-12-17T12:09:23.520483+0000","last_clean":"2023-12-17T12:09:23.520483+0000","last_became_active":"2023-12-17T12:08:49.194315+0000","last_became_peered":"2023-12-17T12:08:49.194315+0000","last_unstale":"2023-12-17T12:09:23.520483+0000","last_undegraded":"2023-12-17T12:09:23.520483+0000","last_fullsized":"2023-12-17T12:09:23.520483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:33:26.056073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517180+0000","last_change":"2023-12-17T12:08:49.196145+0000","last_active":"2023-12-17T12:09:23.517180+0000","last_peered":"2023-12-17T12:09:23.517180+0000","last_clean":"2023-12-17T12:09:23.517180+0000","last_became_active":"2023-12-17T12:08:49.195893+0000","last_became_peered":"2023-12-17T12:08:49.195893+0000","last_unstale":"2023-12-17T12:09:23.517180+0000","last_undegraded":"2023-12-17T12:09:23.517180+0000","last_fullsized":"2023-12-17T12:09:23.517180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:33:23.868301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519244+0000","last_change":"2023-12-17T12:08:49.193113+0000","last_active":"2023-12-17T12:09:23.519244+0000","last_peered":"2023-12-17T12:09:23.519244+0000","last_clean":"2023-12-17T12:09:23.519244+0000","last_became_active":"2023-12-17T12:08:49.191730+0000","last_became_peered":"2023-12-17T12:08:49.191730+0000","last_unstale":"2023-12-17T12:09:23.519244+0000","last_undegraded":"2023-12-17T12:09:23.519244+0000","last_fullsized":"2023-12-17T12:09:23.519244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:45:33.858040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511007+0000","last_change":"2023-12-17T12:08:49.193401+0000","last_active":"2023-12-17T12:09:23.511007+0000","last_peered":"2023-12-17T12:09:23.511007+0000","last_clean":"2023-12-17T12:09:23.511007+0000","last_became_active":"2023-12-17T12:08:49.192993+0000","last_became_peered":"2023-12-17T12:08:49.192993+0000","last_unstale":"2023-12-17T12:09:23.511007+0000","last_undegraded":"2023-12-17T12:09:23.511007+0000","last_fullsized":"2023-12-17T12:09:23.511007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:21:16.594494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.513097+0000","last_change":"2023-12-17T12:08:49.200282+0000","last_active":"2023-12-17T12:09:23.513097+0000","last_peered":"2023-12-17T12:09:23.513097+0000","last_clean":"2023-12-17T12:09:23.513097+0000","last_became_active":"2023-12-17T12:08:49.199788+0000","last_became_peered":"2023-12-17T12:08:49.199788+0000","last_unstale":"2023-12-17T12:09:23.513097+0000","last_undegraded":"2023-12-17T12:09:23.513097+0000","last_fullsized":"2023-12-17T12:09:23.513097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:07:19.164767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515160+0000","last_change":"2023-12-17T12:08:49.185712+0000","last_active":"2023-12-17T12:09:23.515160+0000","last_peered":"2023-12-17T12:09:23.515160+0000","last_clean":"2023-12-17T12:09:23.515160+0000","last_became_active":"2023-12-17T12:08:49.185417+0000","last_became_peered":"2023-12-17T12:08:49.185417+0000","last_unstale":"2023-12-17T12:09:23.515160+0000","last_undegraded":"2023-12-17T12:09:23.515160+0000","last_fullsized":"2023-12-17T12:09:23.515160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:28:43.076574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246978,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61803,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":256,"num_read_kb":193,"num_write":147,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":58386,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":185409,"data_compressed":9050,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481813,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":441806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.755},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58599999999999997}]},{"osd":1,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.627},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.71499999999999997}]},{"osd":5,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.89300000000000002}]},{"osd":6,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.878}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":9,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":1.0589999999999999}]},{"osd":10,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.82199999999999995}]}]},{"osd":10,"up_from":69,"seq":296352743449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1091486,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.436}]},{"osd":2,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.61599999999999999}]},{"osd":3,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.503},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63400000000000001}]},{"osd":4,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.83099999999999996}]},{"osd":6,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.82599999999999996}]},{"osd":8,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.78700000000000003}]},{"osd":9,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58999999999999997}]},{"osd":11,"last update":"Sun Dec 17 12:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.88200000000000001}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":442165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.503},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64100000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.117,"15min":1.117},"last":0.627},{"interface":"front","average":{"1min":0.753,"5min":0.83699999999999997,"15min":0.85099999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.72999999999999998}]},{"osd":2,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.123,"5min":1.306,"15min":1.306},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":1.462,"15min":1.5780000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.097,"5min":13.166,"15min":13.166},"last":0.84899999999999998}]},{"osd":3,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84199999999999997,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.85099999999999998,"15min":0.88100000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":1.917,"15min":1.917},"last":0.76400000000000001}]},{"osd":4,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.91900000000000004,"15min":0.94299999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0549999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93000000000000005,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":1.232,"15min":1.302},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.135,"5min":7.1920000000000002,"15min":7.1920000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.88700000000000001,"15min":0.90100000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.99099999999999999}]},{"osd":6,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":1.0189999999999999,"15min":1.0549999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.048,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.875,"15min":0.89700000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.0780000000000001,"15min":1.1100000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.208,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.1020000000000001,"15min":1.137},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.214,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.86699999999999999}]},{"osd":8,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.071,"15min":1.097},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3320000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.6679999999999999,"15min":1.7929999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.274,"5min":13.18,"15min":13.18},"last":0.96899999999999997}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.745},{"interface":"front","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":13.212,"5min":13.212,"15min":13.212},"last":0.78800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":8,"up_from":56,"seq":240518168611,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":441380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85799999999999998,"5min":1.091,"15min":1.091},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.98499999999999999,"15min":1.0509999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79000000000000004,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.623}]},{"osd":1,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2050000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.748,"5min":1.0509999999999999,"15min":1.101},"min":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.135,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":0.55400000000000005}]},{"osd":2,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.68400000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.96699999999999997,"5min":1.048,"15min":1.048},"last":0.81599999999999995}]},{"osd":3,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":1.079,"15min":1.151},"min":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96299999999999997,"5min":7.0599999999999996,"15min":7.0599999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":1.1259999999999999,"15min":1.1950000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.097,"5min":6.992,"15min":6.992},"last":0.83799999999999997}]},{"osd":4,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":1.087,"15min":1.149},"min":{"1min":0.54200000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.179,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.95899999999999996}]},{"osd":5,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.79400000000000004,"15min":0.80400000000000005},"min":{"1min":0.46100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.0189999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.53300000000000003}]},{"osd":6,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.88700000000000001,"15min":0.89700000000000002},"min":{"1min":0.628,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.018},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.95099999999999996,"15min":0.96499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.276,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92900000000000005,"15min":0.93100000000000005},"min":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.66000000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.73699999999999999}]},{"osd":9,"last update":"Sun Dec 17 12:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.90500000000000003,"15min":0.89500000000000002},"min":{"1min":0.70399999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.94199999999999995,"15min":0.92800000000000005},"min":{"1min":0.80100000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.86699999999999999}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":7,"up_from":49,"seq":210453397543,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":1033565,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92200000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57599999999999996}]},{"osd":1,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.496,"15min":0.48099999999999998},"min":{"1min":0.38900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.74299999999999999}]},{"osd":2,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.78900000000000003,"15min":0.745},"min":{"1min":0.52800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":7.1070000000000002,"5min":7.1070000000000002,"15min":7.1070000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.91200000000000003}]},{"osd":3,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95899999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68899999999999995}]},{"osd":4,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94999999999999996,"5min":1.119,"15min":1.119},"last":0.65300000000000002}]},{"osd":5,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.84899999999999998,"15min":0.80100000000000005},"min":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":7.1669999999999998,"5min":7.1669999999999998,"15min":7.1669999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.96299999999999997}]},{"osd":6,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.84199999999999997,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0780000000000001,"5min":1.21,"15min":1.21},"last":0.73099999999999998}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.69799999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.143,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.012},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.125},"min":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.101,"5min":7.1369999999999996,"15min":7.1369999999999996},"last":0.82299999999999995}]},{"osd":9,"last update":"Sun Dec 17 12:09:22 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":0.96699999999999997,"15min":0.89900000000000002},"min":{"1min":0.68500000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":8.6199999999999992,"5min":8.6199999999999992,"15min":8.6199999999999992},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85299999999999998,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.0529999999999999}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.0940000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]}]},{"osd":6,"up_from":44,"seq":188978561069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":441437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65200000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68500000000000005,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.754,"15min":0.754},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.66000000000000003}]},{"osd":2,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72999999999999998,"15min":0.72699999999999998},"min":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86599999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3540000000000001,"5min":5.6740000000000004,"15min":5.6740000000000004},"last":0.83899999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.621},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":0.44}]},{"osd":4,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":1.1160000000000001,"5min":1.167,"15min":1.167},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.879,"15min":0.86899999999999999},"min":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.1880000000000002,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.75700000000000001}]},{"osd":7,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.749},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.623}]},{"osd":8,"last update":"Sun Dec 17 12:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.99399999999999999,"15min":1.008},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1970000000000001,"5min":1.419,"15min":1.419},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.95199999999999996,"15min":0.96599999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.214,"5min":1.298,"15min":1.298},"last":0.91900000000000004}]},{"osd":9,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.88500000000000001,"15min":0.88900000000000001},"min":{"1min":0.70699999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.236,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94999999999999996,"15min":0.96099999999999997},"min":{"1min":0.57599999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1399999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.68899999999999995}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.70999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":499610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.61799999999999999,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64300000000000002,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":1.032,"15min":1.032},"last":0.58699999999999997}]},{"osd":2,"last update":"Sun Dec 17 12:08:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63800000000000001,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0549999999999999,"5min":1.111,"15min":1.111},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63400000000000001,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.097,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.036}]},{"osd":3,"last update":"Sun Dec 17 12:09:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.65300000000000002,"15min":0.60199999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.65900000000000003,"15min":0.60199999999999998},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.119,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.91800000000000004}]},{"osd":4,"last update":"Sun Dec 17 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.50700000000000001},"min":{"1min":0.38300000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.94299999999999995,"5min":1.01,"15min":1.01},"last":0.81999999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.69699999999999995,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.83799999999999997}]},{"osd":6,"last update":"Sun Dec 17 12:09:22 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.75800000000000001,"15min":0.72599999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.77500000000000002,"15min":0.73999999999999999},"min":{"1min":0.54300000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1439999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Dec 17 12:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68899999999999995,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79000000000000004}]},{"osd":8,"last update":"Sun Dec 17 12:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.77400000000000002,"15min":0.75700000000000001},"min":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.153},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.84099999999999997,"15min":0.81999999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.127}]},{"osd":9,"last update":"Sun Dec 17 12:09:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.879},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.879,"15min":0.871},"min":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.177,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1859999999999999}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.94199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":442702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Dec 17 12:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.61399999999999999},"min":{"1min":0.442,"5min":0.372,"15min":0.372},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67900000000000005,"15min":0.63500000000000001},"min":{"1min":0.42899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Sun Dec 17 12:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65900000000000003,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.747},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65000000000000002,"15min":0.59999999999999998},"min":{"1min":0.55900000000000005,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.72799999999999998}]},{"osd":3,"last update":"Sun Dec 17 12:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.375,"5min":0.315,"15min":0.315},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.505},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69499999999999995}]},{"osd":4,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.64500000000000002},"min":{"1min":0.504,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.077,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:08:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78200000000000003,"15min":0.75},"min":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0680000000000001,"5min":1.248,"15min":1.248},"last":0.92000000000000004}]},{"osd":6,"last update":"Sun Dec 17 12:09:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.367,"5min":2.367,"15min":2.367},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70799999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.83399999999999996},"min":{"1min":0.71999999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.871,"15min":0.86299999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.81599999999999995}]},{"osd":8,"last update":"Sun Dec 17 12:08:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85399999999999998,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.95899999999999996},"min":{"1min":0.753,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.1100000000000001,"5min":1.413,"15min":1.413},"last":0.84099999999999997}]},{"osd":9,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.27,"5min":1.321,"15min":1.321},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.88200000000000001,"15min":0.89500000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.76300000000000001}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80100000000000005}]},{"osd":11,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.90000000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":442574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.52400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.48199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.626}]},{"osd":1,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.66800000000000004,"15min":0.621},"min":{"1min":0.54500000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98099999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.95599999999999996}]},{"osd":3,"last update":"Sun Dec 17 12:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90100000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":0.67100000000000004}]},{"osd":4,"last update":"Sun Dec 17 12:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.69199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95799999999999996,"5min":1.181,"15min":1.181},"last":0.90200000000000002}]},{"osd":5,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94699999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Dec 17 12:09:17 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.79700000000000004,"15min":0.72899999999999998},"min":{"1min":0.73099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.016,"5min":1.214,"15min":1.214},"last":0.84199999999999997}]},{"osd":7,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.75900000000000001}]},{"osd":8,"last update":"Sun Dec 17 12:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80600000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.82699999999999996,"15min":0.82399999999999995},"min":{"1min":0.63,"5min":0.501,"15min":0.501},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0369999999999999}]},{"osd":9,"last update":"Sun Dec 17 12:09:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93500000000000005,"15min":0.93400000000000005},"min":{"1min":0.78600000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.92200000000000004}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.1060000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":441631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.44900000000000001,"15min":0.40300000000000002},"min":{"1min":0.33700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.47599999999999998,"15min":0.42799999999999999},"min":{"1min":0.37,"5min":0.31,"15min":0.31},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70399999999999996}]},{"osd":1,"last update":"Sun Dec 17 12:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.69399999999999995,"15min":0.626},"min":{"1min":0.49399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.67300000000000004,"15min":0.623},"min":{"1min":0.54400000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91100000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.054}]},{"osd":2,"last update":"Sun Dec 17 12:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.623},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71799999999999997,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Sun Dec 17 12:08:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.67500000000000004,"15min":0.63500000000000001},"min":{"1min":0.623,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.014,"5min":1.028,"15min":1.028},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.73399999999999999},"min":{"1min":0.53300000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0820000000000001,"5min":1.278,"15min":1.278},"last":0.80100000000000005}]},{"osd":6,"last update":"Sun Dec 17 12:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93700000000000006,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.72199999999999998,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1279999999999999,"5min":1.429,"15min":1.429},"last":0.51900000000000002}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84999999999999998,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.92600000000000005}]},{"osd":8,"last update":"Sun Dec 17 12:07:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.71999999999999997}]},{"osd":9,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.65000000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.107,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.873},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.91000000000000003,"15min":0.91700000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.96499999999999997}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0029999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":500061,"data_compressed":9050,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.47399999999999998},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48599999999999999,"15min":0.44400000000000001},"min":{"1min":0.38900000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.621}]},{"osd":2,"last update":"Sun Dec 17 12:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76400000000000001,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95499999999999996,"5min":5.125,"15min":5.125},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68500000000000005,"15min":0.64600000000000002},"min":{"1min":0.56699999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95199999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.94299999999999995}]},{"osd":3,"last update":"Sun Dec 17 12:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66600000000000004,"15min":0.624},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96599999999999997,"5min":1.139,"15min":1.139},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67000000000000004,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96599999999999997,"5min":1.222,"15min":1.222},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Dec 17 12:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68400000000000005,"15min":0.65900000000000003},"min":{"1min":0.59599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68799999999999994,"15min":0.66800000000000004},"min":{"1min":0.57599999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90600000000000003,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0089999999999999}]},{"osd":6,"last update":"Sun Dec 17 12:09:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.89500000000000002,"15min":0.95299999999999996},"min":{"1min":0.59699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":0.997,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.57999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0649999999999999,"5min":1.278,"15min":1.278},"last":0.84999999999999998}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.70299999999999996,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.83899999999999997}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.036,"5min":1.339,"15min":1.339},"last":1.093},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.53600000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.03,"5min":1.325,"15min":1.325},"last":0.97499999999999998}]},{"osd":9,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.66200000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1240000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.86899999999999999,"15min":0.86199999999999999},"min":{"1min":0.74199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.056,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.93200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]}]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":1033191,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94999999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.88900000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.55100000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89400000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66200000000000003,"15min":0.63500000000000001},"min":{"1min":0.53400000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.876,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.39700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76900000000000002}]},{"osd":3,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95099999999999996,"5min":1.097,"15min":1.097},"last":0.59599999999999997}]},{"osd":4,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":0.92700000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.52600000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95499999999999996,"5min":1.048,"15min":1.048},"last":0.75}]},{"osd":6,"last update":"Sun Dec 17 12:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78400000000000003,"15min":0.77100000000000002},"min":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0309999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79900000000000004,"15min":0.78500000000000003},"min":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97899999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":1.0589999999999999,"5min":1.117,"15min":1.117},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":0.83899999999999997},"min":{"1min":0.63500000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.95899999999999996,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66400000000000003}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.105,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.83999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1419999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.90200000000000002}]},{"osd":9,"last update":"Sun Dec 17 12:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.79300000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.085,"5min":1.232,"15min":1.232},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.123,"5min":1.137,"15min":1.137},"last":0.875}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.95099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34109,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34109,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":58837,"data_compressed":9050,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-17T12:09:38.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-17T12:09:39.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:38 smithi141 ceph-mon[41225]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:39.778 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:39 smithi027 ceph-mon[36871]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:40.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:39 smithi141 ceph-mon[41225]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:40.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:39 smithi192 ceph-mon[41899]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.508+0000 7fab126ed700 1 -- 172.21.15.27:0/2677742538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c074320 msgr2=0x7fab0c10d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.508+0000 7fab126ed700 1 --2- 172.21.15.27:0/2677742538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c074320 0x7fab0c10d6d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faafc009990 tx=0x7faafc009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.509+0000 7fab126ed700 1 -- 172.21.15.27:0/2677742538 shutdown_connections 2023-12-17T12:09:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.509+0000 7fab126ed700 1 --2- 172.21.15.27:0/2677742538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab0c073a00 0x7fab0c073de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.509+0000 7fab126ed700 1 --2- 172.21.15.27:0/2677742538 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab0c10dc10 0x7fab0c110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.509+0000 7fab126ed700 1 --2- 172.21.15.27:0/2677742538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c074320 0x7fab0c10d6d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.509+0000 7fab126ed700 1 -- 172.21.15.27:0/2677742538 >> 172.21.15.27:0/2677742538 conn(0x7fab0c0fcf00 msgr2=0x7fab0c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.509+0000 7fab126ed700 1 -- 172.21.15.27:0/2677742538 shutdown_connections 2023-12-17T12:09:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.509+0000 7fab126ed700 1 -- 172.21.15.27:0/2677742538 wait complete. 2023-12-17T12:09:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.510+0000 7fab126ed700 1 Processor -- start 2023-12-17T12:09:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.510+0000 7fab126ed700 1 -- start start 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.510+0000 7fab126ed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c073a00 0x7fab0c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.510+0000 7fab126ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab0c074320 0x7fab0c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.510+0000 7fab126ed700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab0c10dc10 0x7fab0c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.510+0000 7fab126ed700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c199e80 con 0x7fab0c073a00 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.510+0000 7fab126ed700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c199ff0 con 0x7fab0c10dc10 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.510+0000 7fab126ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c19a1b0 con 0x7fab0c074320 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c073a00 0x7fab0c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c073a00 0x7fab0c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43662/0 (socket says 172.21.15.27:43662) 2023-12-17T12:09:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 -- 172.21.15.27:0/195744961 learned_addr learned my addr 172.21.15.27:0/195744961 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab10eea700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab0c074320 0x7fab0c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab11eec700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab0c10dc10 0x7fab0c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 -- 172.21.15.27:0/195744961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab0c074320 msgr2=0x7fab0c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab0c074320 0x7fab0c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 -- 172.21.15.27:0/195744961 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab0c10dc10 msgr2=0x7fab0c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab0c10dc10 0x7fab0c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 -- 172.21.15.27:0/195744961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0095f0 con 0x7fab0c073a00 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab11eec700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab0c10dc10 0x7fab0c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.511+0000 7fab116eb700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c073a00 0x7fab0c19f6d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab0800e9a0 tx=0x7fab0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.512+0000 7fab027fc700 1 -- 172.21.15.27:0/195744961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08009790 con 0x7fab0c073a00 2023-12-17T12:09:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.512+0000 7fab027fc700 1 -- 172.21.15.27:0/195744961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab0800cbd0 con 0x7fab0c073a00 2023-12-17T12:09:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.512+0000 7fab027fc700 1 -- 172.21.15.27:0/195744961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08018680 con 0x7fab0c073a00 2023-12-17T12:09:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.512+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0c19a490 con 0x7fab0c073a00 2023-12-17T12:09:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.512+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c1a66c0 con 0x7fab0c073a00 2023-12-17T12:09:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.513+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c108e20 con 0x7fab0c073a00 2023-12-17T12:09:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.515+0000 7fab027fc700 1 -- 172.21.15.27:0/195744961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fab0800cd40 con 0x7fab0c073a00 2023-12-17T12:09:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.516+0000 7fab027fc700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faaf80a1510 0x7faaf80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.516+0000 7fab027fc700 1 -- 172.21.15.27:0/195744961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fab08014070 con 0x7fab0c073a00 2023-12-17T12:09:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.517+0000 7fab10eea700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faaf80a1510 0x7faaf80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.517+0000 7fab10eea700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faaf80a1510 0x7faaf80a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab0c19ac30 tx=0x7faafc00b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.520+0000 7fab027fc700 1 -- 172.21.15.27:0/195744961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fab080522f0 con 0x7fab0c073a00 2023-12-17T12:09:40.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.729+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab0c19acd0 con 0x7faaf80a1510 2023-12-17T12:09:40.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.740+0000 7fab027fc700 1 -- 172.21.15.27:0/195744961 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339425 (secure 0 0 0) 0x7fab0c19acd0 con 0x7faaf80a1510 2023-12-17T12:09:40.745 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:40.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faaf80a1510 msgr2=0x7faaf80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:40.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faaf80a1510 0x7faaf80a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab0c19ac30 tx=0x7faafc00b260 comp rx=0 tx=0).stop 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c073a00 msgr2=0x7fab0c19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c073a00 0x7fab0c19f6d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab0800e9a0 tx=0x7fab0800ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 shutdown_connections 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faaf80a1510 0x7faaf80a39d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab0c10dc10 0x7fab0c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0c073a00 0x7fab0c19f6d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 --2- 172.21.15.27:0/195744961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab0c074320 0x7fab0c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 >> 172.21.15.27:0/195744961 conn(0x7fab0c0fcf00 msgr2=0x7fab0c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 shutdown_connections 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:40.745+0000 7fab126ed700 1 -- 172.21.15.27:0/195744961 wait complete. 2023-12-17T12:09:40.749 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-17T12:09:40.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:40 smithi027 ceph-mon[36871]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:41.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:40 smithi141 ceph-mon[41225]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:41.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:40 smithi192 ceph-mon[41899]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4.8 KiB/s, 0 keys/s, 1 objects/s recovering 2023-12-17T12:09:41.186 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-12-17T12:09:38.807059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652417,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":397,"num_read_kb":312,"num_write":330,"num_write_kb":3041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1239245,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335388,"kb_used_data":11128,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606436,"statfs":{"total":1151940427776,"available":1151596990464,"internally_reserved":0,"allocated":11395072,"data_stored":7251608,"data_compressed":116306,"data_compressed_allocated":147456,"data_compressed_original":1822720,"omap_allocated":19139,"internal_metadata":331855165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58509,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003132"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515969+0000","last_change":"2023-12-17T12:08:49.184999+0000","last_active":"2023-12-17T12:09:23.515969+0000","last_peered":"2023-12-17T12:09:23.515969+0000","last_clean":"2023-12-17T12:09:23.515969+0000","last_became_active":"2023-12-17T12:08:49.184214+0000","last_became_peered":"2023-12-17T12:08:49.184214+0000","last_unstale":"2023-12-17T12:09:23.515969+0000","last_undegraded":"2023-12-17T12:09:23.515969+0000","last_fullsized":"2023-12-17T12:09:23.515969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:48:56.174610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518792+0000","last_change":"2023-12-17T12:08:49.188212+0000","last_active":"2023-12-17T12:09:23.518792+0000","last_peered":"2023-12-17T12:09:23.518792+0000","last_clean":"2023-12-17T12:09:23.518792+0000","last_became_active":"2023-12-17T12:08:49.187869+0000","last_became_peered":"2023-12-17T12:08:49.187869+0000","last_unstale":"2023-12-17T12:09:23.518792+0000","last_undegraded":"2023-12-17T12:09:23.518792+0000","last_fullsized":"2023-12-17T12:09:23.518792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:53:52.485169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519450+0000","last_change":"2023-12-17T12:08:49.182801+0000","last_active":"2023-12-17T12:09:23.519450+0000","last_peered":"2023-12-17T12:09:23.519450+0000","last_clean":"2023-12-17T12:09:23.519450+0000","last_became_active":"2023-12-17T12:08:49.182488+0000","last_became_peered":"2023-12-17T12:08:49.182488+0000","last_unstale":"2023-12-17T12:09:23.519450+0000","last_undegraded":"2023-12-17T12:09:23.519450+0000","last_fullsized":"2023-12-17T12:09:23.519450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:16:43.566881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517285+0000","last_change":"2023-12-17T12:08:49.189279+0000","last_active":"2023-12-17T12:09:23.517285+0000","last_peered":"2023-12-17T12:09:23.517285+0000","last_clean":"2023-12-17T12:09:23.517285+0000","last_became_active":"2023-12-17T12:08:49.186419+0000","last_became_peered":"2023-12-17T12:08:49.186419+0000","last_unstale":"2023-12-17T12:09:23.517285+0000","last_undegraded":"2023-12-17T12:09:23.517285+0000","last_fullsized":"2023-12-17T12:09:23.517285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:47:03.035599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511477+0000","last_change":"2023-12-17T12:08:49.192587+0000","last_active":"2023-12-17T12:09:23.511477+0000","last_peered":"2023-12-17T12:09:23.511477+0000","last_clean":"2023-12-17T12:09:23.511477+0000","last_became_active":"2023-12-17T12:08:49.192272+0000","last_became_peered":"2023-12-17T12:08:49.192272+0000","last_unstale":"2023-12-17T12:09:23.511477+0000","last_undegraded":"2023-12-17T12:09:23.511477+0000","last_fullsized":"2023-12-17T12:09:23.511477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:45:30.131065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514841+0000","last_change":"2023-12-17T12:08:49.176906+0000","last_active":"2023-12-17T12:09:23.514841+0000","last_peered":"2023-12-17T12:09:23.514841+0000","last_clean":"2023-12-17T12:09:23.514841+0000","last_became_active":"2023-12-17T12:08:49.176591+0000","last_became_peered":"2023-12-17T12:08:49.176591+0000","last_unstale":"2023-12-17T12:09:23.514841+0000","last_undegraded":"2023-12-17T12:09:23.514841+0000","last_fullsized":"2023-12-17T12:09:23.514841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:27:17.508628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515618+0000","last_change":"2023-12-17T12:08:49.177409+0000","last_active":"2023-12-17T12:09:23.515618+0000","last_peered":"2023-12-17T12:09:23.515618+0000","last_clean":"2023-12-17T12:09:23.515618+0000","last_became_active":"2023-12-17T12:08:49.177093+0000","last_became_peered":"2023-12-17T12:08:49.177093+0000","last_unstale":"2023-12-17T12:09:23.515618+0000","last_undegraded":"2023-12-17T12:09:23.515618+0000","last_fullsized":"2023-12-17T12:09:23.515618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:56:30.669406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519105+0000","last_change":"2023-12-17T12:08:49.194035+0000","last_active":"2023-12-17T12:09:23.519105+0000","last_peered":"2023-12-17T12:09:23.519105+0000","last_clean":"2023-12-17T12:09:23.519105+0000","last_became_active":"2023-12-17T12:08:49.191677+0000","last_became_peered":"2023-12-17T12:08:49.191677+0000","last_unstale":"2023-12-17T12:09:23.519105+0000","last_undegraded":"2023-12-17T12:09:23.519105+0000","last_fullsized":"2023-12-17T12:09:23.519105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:21:00.984977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511867+0000","last_change":"2023-12-17T12:08:49.187140+0000","last_active":"2023-12-17T12:09:23.511867+0000","last_peered":"2023-12-17T12:09:23.511867+0000","last_clean":"2023-12-17T12:09:23.511867+0000","last_became_active":"2023-12-17T12:08:49.186807+0000","last_became_peered":"2023-12-17T12:08:49.186807+0000","last_unstale":"2023-12-17T12:09:23.511867+0000","last_undegraded":"2023-12-17T12:09:23.511867+0000","last_fullsized":"2023-12-17T12:09:23.511867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:28:59.060986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514453+0000","last_change":"2023-12-17T12:08:49.186371+0000","last_active":"2023-12-17T12:09:23.514453+0000","last_peered":"2023-12-17T12:09:23.514453+0000","last_clean":"2023-12-17T12:09:23.514453+0000","last_became_active":"2023-12-17T12:08:49.186045+0000","last_became_peered":"2023-12-17T12:08:49.186045+0000","last_unstale":"2023-12-17T12:09:23.514453+0000","last_undegraded":"2023-12-17T12:09:23.514453+0000","last_fullsized":"2023-12-17T12:09:23.514453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:56:47.851617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517074+0000","last_change":"2023-12-17T12:08:49.179088+0000","last_active":"2023-12-17T12:09:23.517074+0000","last_peered":"2023-12-17T12:09:23.517074+0000","last_clean":"2023-12-17T12:09:23.517074+0000","last_became_active":"2023-12-17T12:08:49.178545+0000","last_became_peered":"2023-12-17T12:08:49.178545+0000","last_unstale":"2023-12-17T12:09:23.517074+0000","last_undegraded":"2023-12-17T12:09:23.517074+0000","last_fullsized":"2023-12-17T12:09:23.517074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:52:58.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517198+0000","last_change":"2023-12-17T12:08:49.187681+0000","last_active":"2023-12-17T12:09:23.517198+0000","last_peered":"2023-12-17T12:09:23.517198+0000","last_clean":"2023-12-17T12:09:23.517198+0000","last_became_active":"2023-12-17T12:08:49.187086+0000","last_became_peered":"2023-12-17T12:08:49.187086+0000","last_unstale":"2023-12-17T12:09:23.517198+0000","last_undegraded":"2023-12-17T12:09:23.517198+0000","last_fullsized":"2023-12-17T12:09:23.517198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:16:55.476924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.513983+0000","last_change":"2023-12-17T12:08:49.202082+0000","last_active":"2023-12-17T12:09:23.513983+0000","last_peered":"2023-12-17T12:09:23.513983+0000","last_clean":"2023-12-17T12:09:23.513983+0000","last_became_active":"2023-12-17T12:08:49.200583+0000","last_became_peered":"2023-12-17T12:08:49.200583+0000","last_unstale":"2023-12-17T12:09:23.513983+0000","last_undegraded":"2023-12-17T12:09:23.513983+0000","last_fullsized":"2023-12-17T12:09:23.513983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:19:25.112916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517610+0000","last_change":"2023-12-17T12:08:49.184442+0000","last_active":"2023-12-17T12:09:23.517610+0000","last_peered":"2023-12-17T12:09:23.517610+0000","last_clean":"2023-12-17T12:09:23.517610+0000","last_became_active":"2023-12-17T12:08:49.184053+0000","last_became_peered":"2023-12-17T12:08:49.184053+0000","last_unstale":"2023-12-17T12:09:23.517610+0000","last_undegraded":"2023-12-17T12:09:23.517610+0000","last_fullsized":"2023-12-17T12:09:23.517610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:38:18.864989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517395+0000","last_change":"2023-12-17T12:08:49.185550+0000","last_active":"2023-12-17T12:09:23.517395+0000","last_peered":"2023-12-17T12:09:23.517395+0000","last_clean":"2023-12-17T12:09:23.517395+0000","last_became_active":"2023-12-17T12:08:49.185190+0000","last_became_peered":"2023-12-17T12:08:49.185190+0000","last_unstale":"2023-12-17T12:09:23.517395+0000","last_undegraded":"2023-12-17T12:09:23.517395+0000","last_fullsized":"2023-12-17T12:09:23.517395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:11:29.821074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.510773+0000","last_change":"2023-12-17T12:08:49.182738+0000","last_active":"2023-12-17T12:09:23.510773+0000","last_peered":"2023-12-17T12:09:23.510773+0000","last_clean":"2023-12-17T12:09:23.510773+0000","last_became_active":"2023-12-17T12:08:49.182451+0000","last_became_peered":"2023-12-17T12:08:49.182451+0000","last_unstale":"2023-12-17T12:09:23.510773+0000","last_undegraded":"2023-12-17T12:09:23.510773+0000","last_fullsized":"2023-12-17T12:09:23.510773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:21:06.788880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515877+0000","last_change":"2023-12-17T12:08:49.182062+0000","last_active":"2023-12-17T12:09:23.515877+0000","last_peered":"2023-12-17T12:09:23.515877+0000","last_clean":"2023-12-17T12:09:23.515877+0000","last_became_active":"2023-12-17T12:08:49.181725+0000","last_became_peered":"2023-12-17T12:08:49.181725+0000","last_unstale":"2023-12-17T12:09:23.515877+0000","last_undegraded":"2023-12-17T12:09:23.515877+0000","last_fullsized":"2023-12-17T12:09:23.515877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:06:26.626520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518680+0000","last_change":"2023-12-17T12:08:49.174849+0000","last_active":"2023-12-17T12:09:23.518680+0000","last_peered":"2023-12-17T12:09:23.518680+0000","last_clean":"2023-12-17T12:09:23.518680+0000","last_became_active":"2023-12-17T12:08:49.174408+0000","last_became_peered":"2023-12-17T12:08:49.174408+0000","last_unstale":"2023-12-17T12:09:23.518680+0000","last_undegraded":"2023-12-17T12:09:23.518680+0000","last_fullsized":"2023-12-17T12:09:23.518680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:36:11.050556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515127+0000","last_change":"2023-12-17T12:08:49.202639+0000","last_active":"2023-12-17T12:09:23.515127+0000","last_peered":"2023-12-17T12:09:23.515127+0000","last_clean":"2023-12-17T12:09:23.515127+0000","last_became_active":"2023-12-17T12:08:49.202400+0000","last_became_peered":"2023-12-17T12:08:49.202400+0000","last_unstale":"2023-12-17T12:09:23.515127+0000","last_undegraded":"2023-12-17T12:09:23.515127+0000","last_fullsized":"2023-12-17T12:09:23.515127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:45:27.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.532321+0000","last_change":"2023-12-17T12:09:24.532321+0000","last_active":"2023-12-17T12:09:24.532321+0000","last_peered":"2023-12-17T12:09:24.532321+0000","last_clean":"2023-12-17T12:09:24.532321+0000","last_became_active":"2023-12-17T12:09:24.521726+0000","last_became_peered":"2023-12-17T12:09:24.521726+0000","last_unstale":"2023-12-17T12:09:24.532321+0000","last_undegraded":"2023-12-17T12:09:24.532321+0000","last_fullsized":"2023-12-17T12:09:24.532321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:28:15.851568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520325+0000","last_change":"2023-12-17T12:08:49.196221+0000","last_active":"2023-12-17T12:09:23.520325+0000","last_peered":"2023-12-17T12:09:23.520325+0000","last_clean":"2023-12-17T12:09:23.520325+0000","last_became_active":"2023-12-17T12:08:49.195908+0000","last_became_peered":"2023-12-17T12:08:49.195908+0000","last_unstale":"2023-12-17T12:09:23.520325+0000","last_undegraded":"2023-12-17T12:09:23.520325+0000","last_fullsized":"2023-12-17T12:09:23.520325+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:28:17.710818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520412+0000","last_change":"2023-12-17T12:08:48.408069+0000","last_active":"2023-12-17T12:09:23.520412+0000","last_peered":"2023-12-17T12:09:23.520412+0000","last_clean":"2023-12-17T12:09:23.520412+0000","last_became_active":"2023-12-17T12:08:48.407209+0000","last_became_peered":"2023-12-17T12:08:48.407209+0000","last_unstale":"2023-12-17T12:09:23.520412+0000","last_undegraded":"2023-12-17T12:09:23.520412+0000","last_fullsized":"2023-12-17T12:09:23.520412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:24:08.191642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519233+0000","last_change":"2023-12-17T12:08:49.179982+0000","last_active":"2023-12-17T12:09:23.519233+0000","last_peered":"2023-12-17T12:09:23.519233+0000","last_clean":"2023-12-17T12:09:23.519233+0000","last_became_active":"2023-12-17T12:08:49.179610+0000","last_became_peered":"2023-12-17T12:08:49.179610+0000","last_unstale":"2023-12-17T12:09:23.519233+0000","last_undegraded":"2023-12-17T12:09:23.519233+0000","last_fullsized":"2023-12-17T12:09:23.519233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:39:13.133693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511984+0000","last_change":"2023-12-17T12:08:48.395609+0000","last_active":"2023-12-17T12:09:23.511984+0000","last_peered":"2023-12-17T12:09:23.511984+0000","last_clean":"2023-12-17T12:09:23.511984+0000","last_became_active":"2023-12-17T12:08:48.393809+0000","last_became_peered":"2023-12-17T12:08:48.393809+0000","last_unstale":"2023-12-17T12:09:23.511984+0000","last_undegraded":"2023-12-17T12:09:23.511984+0000","last_fullsized":"2023-12-17T12:09:23.511984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:45:39.698185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.510850+0000","last_change":"2023-12-17T12:08:49.182636+0000","last_active":"2023-12-17T12:09:23.510850+0000","last_peered":"2023-12-17T12:09:23.510850+0000","last_clean":"2023-12-17T12:09:23.510850+0000","last_became_active":"2023-12-17T12:08:49.182328+0000","last_became_peered":"2023-12-17T12:08:49.182328+0000","last_unstale":"2023-12-17T12:09:23.510850+0000","last_undegraded":"2023-12-17T12:09:23.510850+0000","last_fullsized":"2023-12-17T12:09:23.510850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:19:27.674053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511750+0000","last_change":"2023-12-17T12:08:48.405333+0000","last_active":"2023-12-17T12:09:23.511750+0000","last_peered":"2023-12-17T12:09:23.511750+0000","last_clean":"2023-12-17T12:09:23.511750+0000","last_became_active":"2023-12-17T12:08:48.403558+0000","last_became_peered":"2023-12-17T12:08:48.403558+0000","last_unstale":"2023-12-17T12:09:23.511750+0000","last_undegraded":"2023-12-17T12:09:23.511750+0000","last_fullsized":"2023-12-17T12:09:23.511750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:16:48.657674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517442+0000","last_change":"2023-12-17T12:08:49.187999+0000","last_active":"2023-12-17T12:09:23.517442+0000","last_peered":"2023-12-17T12:09:23.517442+0000","last_clean":"2023-12-17T12:09:23.517442+0000","last_became_active":"2023-12-17T12:08:49.187698+0000","last_became_peered":"2023-12-17T12:08:49.187698+0000","last_unstale":"2023-12-17T12:09:23.517442+0000","last_undegraded":"2023-12-17T12:09:23.517442+0000","last_fullsized":"2023-12-17T12:09:23.517442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:52:22.279227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":66,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:37.502645+0000","last_change":"2023-12-17T12:08:48.403205+0000","last_active":"2023-12-17T12:09:37.502645+0000","last_peered":"2023-12-17T12:09:37.502645+0000","last_clean":"2023-12-17T12:09:37.502645+0000","last_became_active":"2023-12-17T12:08:48.401930+0000","last_became_peered":"2023-12-17T12:08:48.401930+0000","last_unstale":"2023-12-17T12:09:37.502645+0000","last_undegraded":"2023-12-17T12:09:37.502645+0000","last_fullsized":"2023-12-17T12:09:37.502645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:28:03.889769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511409+0000","last_change":"2023-12-17T12:08:49.176597+0000","last_active":"2023-12-17T12:09:23.511409+0000","last_peered":"2023-12-17T12:09:23.511409+0000","last_clean":"2023-12-17T12:09:23.511409+0000","last_became_active":"2023-12-17T12:08:49.176279+0000","last_became_peered":"2023-12-17T12:08:49.176279+0000","last_unstale":"2023-12-17T12:09:23.511409+0000","last_undegraded":"2023-12-17T12:09:23.511409+0000","last_fullsized":"2023-12-17T12:09:23.511409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:38:48.558555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514043+0000","last_change":"2023-12-17T12:08:48.390768+0000","last_active":"2023-12-17T12:09:23.514043+0000","last_peered":"2023-12-17T12:09:23.514043+0000","last_clean":"2023-12-17T12:09:23.514043+0000","last_became_active":"2023-12-17T12:08:48.390452+0000","last_became_peered":"2023-12-17T12:08:48.390452+0000","last_unstale":"2023-12-17T12:09:23.514043+0000","last_undegraded":"2023-12-17T12:09:23.514043+0000","last_fullsized":"2023-12-17T12:09:23.514043+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:16:50.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516911+0000","last_change":"2023-12-17T12:08:49.194554+0000","last_active":"2023-12-17T12:09:23.516911+0000","last_peered":"2023-12-17T12:09:23.516911+0000","last_clean":"2023-12-17T12:09:23.516911+0000","last_became_active":"2023-12-17T12:08:49.193794+0000","last_became_peered":"2023-12-17T12:08:49.193794+0000","last_unstale":"2023-12-17T12:09:23.516911+0000","last_undegraded":"2023-12-17T12:09:23.516911+0000","last_fullsized":"2023-12-17T12:09:23.516911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:19:30.140621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511983+0000","last_change":"2023-12-17T12:08:48.388949+0000","last_active":"2023-12-17T12:09:23.511983+0000","last_peered":"2023-12-17T12:09:23.511983+0000","last_clean":"2023-12-17T12:09:23.511983+0000","last_became_active":"2023-12-17T12:08:48.386904+0000","last_became_peered":"2023-12-17T12:08:48.386904+0000","last_unstale":"2023-12-17T12:09:23.511983+0000","last_undegraded":"2023-12-17T12:09:23.511983+0000","last_fullsized":"2023-12-17T12:09:23.511983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:25:42.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519088+0000","last_change":"2023-12-17T12:08:49.197069+0000","last_active":"2023-12-17T12:09:23.519088+0000","last_peered":"2023-12-17T12:09:23.519088+0000","last_clean":"2023-12-17T12:09:23.519088+0000","last_became_active":"2023-12-17T12:08:49.196366+0000","last_became_peered":"2023-12-17T12:08:49.196366+0000","last_unstale":"2023-12-17T12:09:23.519088+0000","last_undegraded":"2023-12-17T12:09:23.519088+0000","last_fullsized":"2023-12-17T12:09:23.519088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:43:41.262255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.522956+0000","last_change":"2023-12-17T12:09:24.522454+0000","last_active":"2023-12-17T12:09:24.522956+0000","last_peered":"2023-12-17T12:09:24.522956+0000","last_clean":"2023-12-17T12:09:24.522956+0000","last_became_active":"2023-12-17T12:09:24.521938+0000","last_became_peered":"2023-12-17T12:09:24.521938+0000","last_unstale":"2023-12-17T12:09:24.522956+0000","last_undegraded":"2023-12-17T12:09:24.522956+0000","last_fullsized":"2023-12-17T12:09:24.522956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:59:54.789618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515897+0000","last_change":"2023-12-17T12:08:49.192837+0000","last_active":"2023-12-17T12:09:23.515897+0000","last_peered":"2023-12-17T12:09:23.515897+0000","last_clean":"2023-12-17T12:09:23.515897+0000","last_became_active":"2023-12-17T12:08:49.192479+0000","last_became_peered":"2023-12-17T12:08:49.192479+0000","last_unstale":"2023-12-17T12:09:23.515897+0000","last_undegraded":"2023-12-17T12:09:23.515897+0000","last_fullsized":"2023-12-17T12:09:23.515897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:08:50.771280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":57,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:32.493884+0000","last_change":"2023-12-17T12:08:48.395661+0000","last_active":"2023-12-17T12:09:32.493884+0000","last_peered":"2023-12-17T12:09:32.493884+0000","last_clean":"2023-12-17T12:09:32.493884+0000","last_became_active":"2023-12-17T12:08:48.395282+0000","last_became_peered":"2023-12-17T12:08:48.395282+0000","last_unstale":"2023-12-17T12:09:32.493884+0000","last_undegraded":"2023-12-17T12:09:32.493884+0000","last_fullsized":"2023-12-17T12:09:32.493884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:02:15.554520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511299+0000","last_change":"2023-12-17T12:08:49.179339+0000","last_active":"2023-12-17T12:09:23.511299+0000","last_peered":"2023-12-17T12:09:23.511299+0000","last_clean":"2023-12-17T12:09:23.511299+0000","last_became_active":"2023-12-17T12:08:49.179041+0000","last_became_peered":"2023-12-17T12:08:49.179041+0000","last_unstale":"2023-12-17T12:09:23.511299+0000","last_undegraded":"2023-12-17T12:09:23.511299+0000","last_fullsized":"2023-12-17T12:09:23.511299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:27:28.508576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517241+0000","last_change":"2023-12-17T12:08:48.401911+0000","last_active":"2023-12-17T12:09:23.517241+0000","last_peered":"2023-12-17T12:09:23.517241+0000","last_clean":"2023-12-17T12:09:23.517241+0000","last_became_active":"2023-12-17T12:08:48.401552+0000","last_became_peered":"2023-12-17T12:08:48.401552+0000","last_unstale":"2023-12-17T12:09:23.517241+0000","last_undegraded":"2023-12-17T12:09:23.517241+0000","last_fullsized":"2023-12-17T12:09:23.517241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:40:54.813575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519095+0000","last_change":"2023-12-17T12:08:49.177683+0000","last_active":"2023-12-17T12:09:23.519095+0000","last_peered":"2023-12-17T12:09:23.519095+0000","last_clean":"2023-12-17T12:09:23.519095+0000","last_became_active":"2023-12-17T12:08:49.177313+0000","last_became_peered":"2023-12-17T12:08:49.177313+0000","last_unstale":"2023-12-17T12:09:23.519095+0000","last_undegraded":"2023-12-17T12:09:23.519095+0000","last_fullsized":"2023-12-17T12:09:23.519095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:29:40.397954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517631+0000","last_change":"2023-12-17T12:08:48.392907+0000","last_active":"2023-12-17T12:09:23.517631+0000","last_peered":"2023-12-17T12:09:23.517631+0000","last_clean":"2023-12-17T12:09:23.517631+0000","last_became_active":"2023-12-17T12:08:48.392504+0000","last_became_peered":"2023-12-17T12:08:48.392504+0000","last_unstale":"2023-12-17T12:09:23.517631+0000","last_undegraded":"2023-12-17T12:09:23.517631+0000","last_fullsized":"2023-12-17T12:09:23.517631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:25:24.561271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511030+0000","last_change":"2023-12-17T12:08:49.186687+0000","last_active":"2023-12-17T12:09:23.511030+0000","last_peered":"2023-12-17T12:09:23.511030+0000","last_clean":"2023-12-17T12:09:23.511030+0000","last_became_active":"2023-12-17T12:08:49.186400+0000","last_became_peered":"2023-12-17T12:08:49.186400+0000","last_unstale":"2023-12-17T12:09:23.511030+0000","last_undegraded":"2023-12-17T12:09:23.511030+0000","last_fullsized":"2023-12-17T12:09:23.511030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:12:27.143793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'25","reported_seq":145,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:34.072478+0000","last_change":"2023-12-17T12:09:25.063100+0000","last_active":"2023-12-17T12:09:34.072478+0000","last_peered":"2023-12-17T12:09:34.072478+0000","last_clean":"2023-12-17T12:09:34.072478+0000","last_became_active":"2023-12-17T12:09:25.051399+0000","last_became_peered":"2023-12-17T12:09:25.051399+0000","last_unstale":"2023-12-17T12:09:34.072478+0000","last_undegraded":"2023-12-17T12:09:34.072478+0000","last_fullsized":"2023-12-17T12:09:34.072478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:17:57.842293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57804,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":77,"num_read_kb":94,"num_write":26,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57804,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511762+0000","last_change":"2023-12-17T12:08:49.176410+0000","last_active":"2023-12-17T12:09:23.511762+0000","last_peered":"2023-12-17T12:09:23.511762+0000","last_clean":"2023-12-17T12:09:23.511762+0000","last_became_active":"2023-12-17T12:08:49.176120+0000","last_became_peered":"2023-12-17T12:08:49.176120+0000","last_unstale":"2023-12-17T12:09:23.511762+0000","last_undegraded":"2023-12-17T12:09:23.511762+0000","last_fullsized":"2023-12-17T12:09:23.511762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:28:18.241319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514401+0000","last_change":"2023-12-17T12:08:48.405781+0000","last_active":"2023-12-17T12:09:23.514401+0000","last_peered":"2023-12-17T12:09:23.514401+0000","last_clean":"2023-12-17T12:09:23.514401+0000","last_became_active":"2023-12-17T12:08:48.405255+0000","last_became_peered":"2023-12-17T12:08:48.405255+0000","last_unstale":"2023-12-17T12:09:23.514401+0000","last_undegraded":"2023-12-17T12:09:23.514401+0000","last_fullsized":"2023-12-17T12:09:23.514401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:00:10.046901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511949+0000","last_change":"2023-12-17T12:08:49.199457+0000","last_active":"2023-12-17T12:09:23.511949+0000","last_peered":"2023-12-17T12:09:23.511949+0000","last_clean":"2023-12-17T12:09:23.511949+0000","last_became_active":"2023-12-17T12:08:49.197147+0000","last_became_peered":"2023-12-17T12:08:49.197147+0000","last_unstale":"2023-12-17T12:09:23.511949+0000","last_undegraded":"2023-12-17T12:09:23.511949+0000","last_fullsized":"2023-12-17T12:09:23.511949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:36:46.619674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517016+0000","last_change":"2023-12-17T12:08:48.407127+0000","last_active":"2023-12-17T12:09:23.517016+0000","last_peered":"2023-12-17T12:09:23.517016+0000","last_clean":"2023-12-17T12:09:23.517016+0000","last_became_active":"2023-12-17T12:08:48.406726+0000","last_became_peered":"2023-12-17T12:08:48.406726+0000","last_unstale":"2023-12-17T12:09:23.517016+0000","last_undegraded":"2023-12-17T12:09:23.517016+0000","last_fullsized":"2023-12-17T12:09:23.517016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:16:19.492428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.523348+0000","last_change":"2023-12-17T12:09:24.523348+0000","last_active":"2023-12-17T12:09:24.523348+0000","last_peered":"2023-12-17T12:09:24.523348+0000","last_clean":"2023-12-17T12:09:24.523348+0000","last_became_active":"2023-12-17T12:09:24.523000+0000","last_became_peered":"2023-12-17T12:09:24.523000+0000","last_unstale":"2023-12-17T12:09:24.523348+0000","last_undegraded":"2023-12-17T12:09:24.523348+0000","last_fullsized":"2023-12-17T12:09:24.523348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:22:21.503203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:34.497607+0000","last_change":"2023-12-17T12:08:48.393092+0000","last_active":"2023-12-17T12:09:34.497607+0000","last_peered":"2023-12-17T12:09:34.497607+0000","last_clean":"2023-12-17T12:09:34.497607+0000","last_became_active":"2023-12-17T12:08:48.392702+0000","last_became_peered":"2023-12-17T12:08:48.392702+0000","last_unstale":"2023-12-17T12:09:34.497607+0000","last_undegraded":"2023-12-17T12:09:34.497607+0000","last_fullsized":"2023-12-17T12:09:34.497607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:53:00.261870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":977,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516437+0000","last_change":"2023-12-17T12:08:49.181367+0000","last_active":"2023-12-17T12:09:23.516437+0000","last_peered":"2023-12-17T12:09:23.516437+0000","last_clean":"2023-12-17T12:09:23.516437+0000","last_became_active":"2023-12-17T12:08:49.180841+0000","last_became_peered":"2023-12-17T12:08:49.180841+0000","last_unstale":"2023-12-17T12:09:23.516437+0000","last_undegraded":"2023-12-17T12:09:23.516437+0000","last_fullsized":"2023-12-17T12:09:23.516437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:57:33.920366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511292+0000","last_change":"2023-12-17T12:08:48.398863+0000","last_active":"2023-12-17T12:09:23.511292+0000","last_peered":"2023-12-17T12:09:23.511292+0000","last_clean":"2023-12-17T12:09:23.511292+0000","last_became_active":"2023-12-17T12:08:48.396178+0000","last_became_peered":"2023-12-17T12:08:48.396178+0000","last_unstale":"2023-12-17T12:09:23.511292+0000","last_undegraded":"2023-12-17T12:09:23.511292+0000","last_fullsized":"2023-12-17T12:09:23.511292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:31:44.593246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516840+0000","last_change":"2023-12-17T12:08:49.190795+0000","last_active":"2023-12-17T12:09:23.516840+0000","last_peered":"2023-12-17T12:09:23.516840+0000","last_clean":"2023-12-17T12:09:23.516840+0000","last_became_active":"2023-12-17T12:08:49.190468+0000","last_became_peered":"2023-12-17T12:08:49.190468+0000","last_unstale":"2023-12-17T12:09:23.516840+0000","last_undegraded":"2023-12-17T12:09:23.516840+0000","last_fullsized":"2023-12-17T12:09:23.516840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:26:37.302852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.512001+0000","last_change":"2023-12-17T12:08:48.391524+0000","last_active":"2023-12-17T12:09:23.512001+0000","last_peered":"2023-12-17T12:09:23.512001+0000","last_clean":"2023-12-17T12:09:23.512001+0000","last_became_active":"2023-12-17T12:08:48.391195+0000","last_became_peered":"2023-12-17T12:08:48.391195+0000","last_unstale":"2023-12-17T12:09:23.512001+0000","last_undegraded":"2023-12-17T12:09:23.512001+0000","last_fullsized":"2023-12-17T12:09:23.512001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:33:55.399168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515717+0000","last_change":"2023-12-17T12:08:49.188376+0000","last_active":"2023-12-17T12:09:23.515717+0000","last_peered":"2023-12-17T12:09:23.515717+0000","last_clean":"2023-12-17T12:09:23.515717+0000","last_became_active":"2023-12-17T12:08:49.188038+0000","last_became_peered":"2023-12-17T12:08:49.188038+0000","last_unstale":"2023-12-17T12:09:23.515717+0000","last_undegraded":"2023-12-17T12:09:23.515717+0000","last_fullsized":"2023-12-17T12:09:23.515717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:27:35.263053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515192+0000","last_change":"2023-12-17T12:08:48.401451+0000","last_active":"2023-12-17T12:09:23.515192+0000","last_peered":"2023-12-17T12:09:23.515192+0000","last_clean":"2023-12-17T12:09:23.515192+0000","last_became_active":"2023-12-17T12:08:48.401100+0000","last_became_peered":"2023-12-17T12:08:48.401100+0000","last_unstale":"2023-12-17T12:09:23.515192+0000","last_undegraded":"2023-12-17T12:09:23.515192+0000","last_fullsized":"2023-12-17T12:09:23.515192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:38:48.372997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514180+0000","last_change":"2023-12-17T12:08:49.194033+0000","last_active":"2023-12-17T12:09:23.514180+0000","last_peered":"2023-12-17T12:09:23.514180+0000","last_clean":"2023-12-17T12:09:23.514180+0000","last_became_active":"2023-12-17T12:08:49.192344+0000","last_became_peered":"2023-12-17T12:08:49.192344+0000","last_unstale":"2023-12-17T12:09:23.514180+0000","last_undegraded":"2023-12-17T12:09:23.514180+0000","last_fullsized":"2023-12-17T12:09:23.514180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:00:40.106655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515945+0000","last_change":"2023-12-17T12:08:48.397060+0000","last_active":"2023-12-17T12:09:23.515945+0000","last_peered":"2023-12-17T12:09:23.515945+0000","last_clean":"2023-12-17T12:09:23.515945+0000","last_became_active":"2023-12-17T12:08:48.396704+0000","last_became_peered":"2023-12-17T12:08:48.396704+0000","last_unstale":"2023-12-17T12:09:23.515945+0000","last_undegraded":"2023-12-17T12:09:23.515945+0000","last_fullsized":"2023-12-17T12:09:23.515945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:31:08.882089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514200+0000","last_change":"2023-12-17T12:08:49.183915+0000","last_active":"2023-12-17T12:09:23.514200+0000","last_peered":"2023-12-17T12:09:23.514200+0000","last_clean":"2023-12-17T12:09:23.514200+0000","last_became_active":"2023-12-17T12:08:49.183615+0000","last_became_peered":"2023-12-17T12:08:49.183615+0000","last_unstale":"2023-12-17T12:09:23.514200+0000","last_undegraded":"2023-12-17T12:09:23.514200+0000","last_fullsized":"2023-12-17T12:09:23.514200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:29:09.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517302+0000","last_change":"2023-12-17T12:08:49.190952+0000","last_active":"2023-12-17T12:09:23.517302+0000","last_peered":"2023-12-17T12:09:23.517302+0000","last_clean":"2023-12-17T12:09:23.517302+0000","last_became_active":"2023-12-17T12:08:49.190586+0000","last_became_peered":"2023-12-17T12:08:49.190586+0000","last_unstale":"2023-12-17T12:09:23.517302+0000","last_undegraded":"2023-12-17T12:09:23.517302+0000","last_fullsized":"2023-12-17T12:09:23.517302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:44:00.972429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:34.497174+0000","last_change":"2023-12-17T12:08:48.409488+0000","last_active":"2023-12-17T12:09:34.497174+0000","last_peered":"2023-12-17T12:09:34.497174+0000","last_clean":"2023-12-17T12:09:34.497174+0000","last_became_active":"2023-12-17T12:08:48.408703+0000","last_became_peered":"2023-12-17T12:08:48.408703+0000","last_unstale":"2023-12-17T12:09:34.497174+0000","last_undegraded":"2023-12-17T12:09:34.497174+0000","last_fullsized":"2023-12-17T12:09:34.497174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:33:14.861530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519847+0000","last_change":"2023-12-17T12:08:49.197960+0000","last_active":"2023-12-17T12:09:23.519847+0000","last_peered":"2023-12-17T12:09:23.519847+0000","last_clean":"2023-12-17T12:09:23.519847+0000","last_became_active":"2023-12-17T12:08:49.196484+0000","last_became_peered":"2023-12-17T12:08:49.196484+0000","last_unstale":"2023-12-17T12:09:23.519847+0000","last_undegraded":"2023-12-17T12:09:23.519847+0000","last_fullsized":"2023-12-17T12:09:23.519847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:53:50.293026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518375+0000","last_change":"2023-12-17T12:08:48.387589+0000","last_active":"2023-12-17T12:09:23.518375+0000","last_peered":"2023-12-17T12:09:23.518375+0000","last_clean":"2023-12-17T12:09:23.518375+0000","last_became_active":"2023-12-17T12:08:48.387258+0000","last_became_peered":"2023-12-17T12:08:48.387258+0000","last_unstale":"2023-12-17T12:09:23.518375+0000","last_undegraded":"2023-12-17T12:09:23.518375+0000","last_fullsized":"2023-12-17T12:09:23.518375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:29:30.815005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516549+0000","last_change":"2023-12-17T12:07:42.894663+0000","last_active":"2023-12-17T12:09:23.516549+0000","last_peered":"2023-12-17T12:09:23.516549+0000","last_clean":"2023-12-17T12:09:23.516549+0000","last_became_active":"2023-12-17T12:07:42.894199+0000","last_became_peered":"2023-12-17T12:07:42.894199+0000","last_unstale":"2023-12-17T12:09:23.516549+0000","last_undegraded":"2023-12-17T12:09:23.516549+0000","last_fullsized":"2023-12-17T12:09:23.516549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:04:41.531220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:04:41.531220+0000","last_clean_scrub_stamp":"2023-12-17T12:04:41.531220+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:07:49.358197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511873+0000","last_change":"2023-12-17T12:08:49.195358+0000","last_active":"2023-12-17T12:09:23.511873+0000","last_peered":"2023-12-17T12:09:23.511873+0000","last_clean":"2023-12-17T12:09:23.511873+0000","last_became_active":"2023-12-17T12:08:49.194996+0000","last_became_peered":"2023-12-17T12:08:49.194996+0000","last_unstale":"2023-12-17T12:09:23.511873+0000","last_undegraded":"2023-12-17T12:09:23.511873+0000","last_fullsized":"2023-12-17T12:09:23.511873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:23:10.016755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516724+0000","last_change":"2023-12-17T12:08:48.394937+0000","last_active":"2023-12-17T12:09:23.516724+0000","last_peered":"2023-12-17T12:09:23.516724+0000","last_clean":"2023-12-17T12:09:23.516724+0000","last_became_active":"2023-12-17T12:08:48.394590+0000","last_became_peered":"2023-12-17T12:08:48.394590+0000","last_unstale":"2023-12-17T12:09:23.516724+0000","last_undegraded":"2023-12-17T12:09:23.516724+0000","last_fullsized":"2023-12-17T12:09:23.516724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:24:49.088883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.512088+0000","last_change":"2023-12-17T12:08:48.396769+0000","last_active":"2023-12-17T12:09:23.512088+0000","last_peered":"2023-12-17T12:09:23.512088+0000","last_clean":"2023-12-17T12:09:23.512088+0000","last_became_active":"2023-12-17T12:08:48.395984+0000","last_became_peered":"2023-12-17T12:08:48.395984+0000","last_unstale":"2023-12-17T12:09:23.512088+0000","last_undegraded":"2023-12-17T12:09:23.512088+0000","last_fullsized":"2023-12-17T12:09:23.512088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:34:59.872771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.512311+0000","last_change":"2023-12-17T12:08:49.194598+0000","last_active":"2023-12-17T12:09:23.512311+0000","last_peered":"2023-12-17T12:09:23.512311+0000","last_clean":"2023-12-17T12:09:23.512311+0000","last_became_active":"2023-12-17T12:08:49.189450+0000","last_became_peered":"2023-12-17T12:08:49.189450+0000","last_unstale":"2023-12-17T12:09:23.512311+0000","last_undegraded":"2023-12-17T12:09:23.512311+0000","last_fullsized":"2023-12-17T12:09:23.512311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:45:57.697614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.535949+0000","last_change":"2023-12-17T12:09:24.535949+0000","last_active":"2023-12-17T12:09:24.535949+0000","last_peered":"2023-12-17T12:09:24.535949+0000","last_clean":"2023-12-17T12:09:24.535949+0000","last_became_active":"2023-12-17T12:09:24.526345+0000","last_became_peered":"2023-12-17T12:09:24.526345+0000","last_unstale":"2023-12-17T12:09:24.535949+0000","last_undegraded":"2023-12-17T12:09:24.535949+0000","last_fullsized":"2023-12-17T12:09:24.535949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:50:30.756017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520562+0000","last_change":"2023-12-17T12:08:49.196489+0000","last_active":"2023-12-17T12:09:23.520562+0000","last_peered":"2023-12-17T12:09:23.520562+0000","last_clean":"2023-12-17T12:09:23.520562+0000","last_became_active":"2023-12-17T12:08:49.196063+0000","last_became_peered":"2023-12-17T12:08:49.196063+0000","last_unstale":"2023-12-17T12:09:23.520562+0000","last_undegraded":"2023-12-17T12:09:23.520562+0000","last_fullsized":"2023-12-17T12:09:23.520562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:16:08.658987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518017+0000","last_change":"2023-12-17T12:08:48.400075+0000","last_active":"2023-12-17T12:09:23.518017+0000","last_peered":"2023-12-17T12:09:23.518017+0000","last_clean":"2023-12-17T12:09:23.518017+0000","last_became_active":"2023-12-17T12:08:48.398728+0000","last_became_peered":"2023-12-17T12:08:48.398728+0000","last_unstale":"2023-12-17T12:09:23.518017+0000","last_undegraded":"2023-12-17T12:09:23.518017+0000","last_fullsized":"2023-12-17T12:09:23.518017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T22:10:21.065038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.518876+0000","last_change":"2023-12-17T12:08:49.193200+0000","last_active":"2023-12-17T12:09:23.518876+0000","last_peered":"2023-12-17T12:09:23.518876+0000","last_clean":"2023-12-17T12:09:23.518876+0000","last_became_active":"2023-12-17T12:08:49.192855+0000","last_became_peered":"2023-12-17T12:08:49.192855+0000","last_unstale":"2023-12-17T12:09:23.518876+0000","last_undegraded":"2023-12-17T12:09:23.518876+0000","last_fullsized":"2023-12-17T12:09:23.518876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:14:09.089956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.510772+0000","last_change":"2023-12-17T12:08:48.404661+0000","last_active":"2023-12-17T12:09:23.510772+0000","last_peered":"2023-12-17T12:09:23.510772+0000","last_clean":"2023-12-17T12:09:23.510772+0000","last_became_active":"2023-12-17T12:08:48.404336+0000","last_became_peered":"2023-12-17T12:08:48.404336+0000","last_unstale":"2023-12-17T12:09:23.510772+0000","last_undegraded":"2023-12-17T12:09:23.510772+0000","last_fullsized":"2023-12-17T12:09:23.510772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:37:20.863966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511268+0000","last_change":"2023-12-17T12:08:49.181169+0000","last_active":"2023-12-17T12:09:23.511268+0000","last_peered":"2023-12-17T12:09:23.511268+0000","last_clean":"2023-12-17T12:09:23.511268+0000","last_became_active":"2023-12-17T12:08:49.177802+0000","last_became_peered":"2023-12-17T12:08:49.177802+0000","last_unstale":"2023-12-17T12:09:23.511268+0000","last_undegraded":"2023-12-17T12:09:23.511268+0000","last_fullsized":"2023-12-17T12:09:23.511268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:56:50.122246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519361+0000","last_change":"2023-12-17T12:08:48.407865+0000","last_active":"2023-12-17T12:09:23.519361+0000","last_peered":"2023-12-17T12:09:23.519361+0000","last_clean":"2023-12-17T12:09:23.519361+0000","last_became_active":"2023-12-17T12:08:48.407580+0000","last_became_peered":"2023-12-17T12:08:48.407580+0000","last_unstale":"2023-12-17T12:09:23.519361+0000","last_undegraded":"2023-12-17T12:09:23.519361+0000","last_fullsized":"2023-12-17T12:09:23.519361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:10:20.505630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515970+0000","last_change":"2023-12-17T12:08:49.192724+0000","last_active":"2023-12-17T12:09:23.515970+0000","last_peered":"2023-12-17T12:09:23.515970+0000","last_clean":"2023-12-17T12:09:23.515970+0000","last_became_active":"2023-12-17T12:08:49.188658+0000","last_became_peered":"2023-12-17T12:08:49.188658+0000","last_unstale":"2023-12-17T12:09:23.515970+0000","last_undegraded":"2023-12-17T12:09:23.515970+0000","last_fullsized":"2023-12-17T12:09:23.515970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T16:00:17.493003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514328+0000","last_change":"2023-12-17T12:08:48.404717+0000","last_active":"2023-12-17T12:09:23.514328+0000","last_peered":"2023-12-17T12:09:23.514328+0000","last_clean":"2023-12-17T12:09:23.514328+0000","last_became_active":"2023-12-17T12:08:48.404426+0000","last_became_peered":"2023-12-17T12:08:48.404426+0000","last_unstale":"2023-12-17T12:09:23.514328+0000","last_undegraded":"2023-12-17T12:09:23.514328+0000","last_fullsized":"2023-12-17T12:09:23.514328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:35:28.090855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520139+0000","last_change":"2023-12-17T12:08:49.191911+0000","last_active":"2023-12-17T12:09:23.520139+0000","last_peered":"2023-12-17T12:09:23.520139+0000","last_clean":"2023-12-17T12:09:23.520139+0000","last_became_active":"2023-12-17T12:08:49.191573+0000","last_became_peered":"2023-12-17T12:08:49.191573+0000","last_unstale":"2023-12-17T12:09:23.520139+0000","last_undegraded":"2023-12-17T12:09:23.520139+0000","last_fullsized":"2023-12-17T12:09:23.520139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:39:35.354221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:25.056633+0000","last_change":"2023-12-17T12:09:25.056633+0000","last_active":"2023-12-17T12:09:25.056633+0000","last_peered":"2023-12-17T12:09:25.056633+0000","last_clean":"2023-12-17T12:09:25.056633+0000","last_became_active":"2023-12-17T12:09:25.051396+0000","last_became_peered":"2023-12-17T12:09:25.051396+0000","last_unstale":"2023-12-17T12:09:25.056633+0000","last_undegraded":"2023-12-17T12:09:25.056633+0000","last_fullsized":"2023-12-17T12:09:25.056633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T20:50:14.091049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.516058+0000","last_change":"2023-12-17T12:08:49.191906+0000","last_active":"2023-12-17T12:09:23.516058+0000","last_peered":"2023-12-17T12:09:23.516058+0000","last_clean":"2023-12-17T12:09:23.516058+0000","last_became_active":"2023-12-17T12:08:49.191572+0000","last_became_peered":"2023-12-17T12:08:49.191572+0000","last_unstale":"2023-12-17T12:09:23.516058+0000","last_undegraded":"2023-12-17T12:09:23.516058+0000","last_fullsized":"2023-12-17T12:09:23.516058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:56:05.670034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515249+0000","last_change":"2023-12-17T12:08:48.399004+0000","last_active":"2023-12-17T12:09:23.515249+0000","last_peered":"2023-12-17T12:09:23.515249+0000","last_clean":"2023-12-17T12:09:23.515249+0000","last_became_active":"2023-12-17T12:08:48.398453+0000","last_became_peered":"2023-12-17T12:08:48.398453+0000","last_unstale":"2023-12-17T12:09:23.515249+0000","last_undegraded":"2023-12-17T12:09:23.515249+0000","last_fullsized":"2023-12-17T12:09:23.515249+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T13:30:53.767008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517280+0000","last_change":"2023-12-17T12:08:49.178859+0000","last_active":"2023-12-17T12:09:23.517280+0000","last_peered":"2023-12-17T12:09:23.517280+0000","last_clean":"2023-12-17T12:09:23.517280+0000","last_became_active":"2023-12-17T12:08:49.178119+0000","last_became_peered":"2023-12-17T12:08:49.178119+0000","last_unstale":"2023-12-17T12:09:23.517280+0000","last_undegraded":"2023-12-17T12:09:23.517280+0000","last_fullsized":"2023-12-17T12:09:23.517280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T23:32:40.758242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517279+0000","last_change":"2023-12-17T12:08:48.407249+0000","last_active":"2023-12-17T12:09:23.517279+0000","last_peered":"2023-12-17T12:09:23.517279+0000","last_clean":"2023-12-17T12:09:23.517279+0000","last_became_active":"2023-12-17T12:08:48.406572+0000","last_became_peered":"2023-12-17T12:08:48.406572+0000","last_unstale":"2023-12-17T12:09:23.517279+0000","last_undegraded":"2023-12-17T12:09:23.517279+0000","last_fullsized":"2023-12-17T12:09:23.517279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:19:07.751233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:24.527971+0000","last_change":"2023-12-17T12:09:24.527971+0000","last_active":"2023-12-17T12:09:24.527971+0000","last_peered":"2023-12-17T12:09:24.527971+0000","last_clean":"2023-12-17T12:09:24.527971+0000","last_became_active":"2023-12-17T12:09:24.523308+0000","last_became_peered":"2023-12-17T12:09:24.523308+0000","last_unstale":"2023-12-17T12:09:24.527971+0000","last_undegraded":"2023-12-17T12:09:24.527971+0000","last_fullsized":"2023-12-17T12:09:24.527971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T19:17:08.560065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.512717+0000","last_change":"2023-12-17T12:08:48.406799+0000","last_active":"2023-12-17T12:09:23.512717+0000","last_peered":"2023-12-17T12:09:23.512717+0000","last_clean":"2023-12-17T12:09:23.512717+0000","last_became_active":"2023-12-17T12:08:48.406423+0000","last_became_peered":"2023-12-17T12:08:48.406423+0000","last_unstale":"2023-12-17T12:09:23.512717+0000","last_undegraded":"2023-12-17T12:09:23.512717+0000","last_fullsized":"2023-12-17T12:09:23.512717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:47.149209+0000","last_clean_scrub_stamp":"2023-12-17T12:08:47.149209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:43:55.600399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517454+0000","last_change":"2023-12-17T12:08:49.182840+0000","last_active":"2023-12-17T12:09:23.517454+0000","last_peered":"2023-12-17T12:09:23.517454+0000","last_clean":"2023-12-17T12:09:23.517454+0000","last_became_active":"2023-12-17T12:08:49.181562+0000","last_became_peered":"2023-12-17T12:08:49.181562+0000","last_unstale":"2023-12-17T12:09:23.517454+0000","last_undegraded":"2023-12-17T12:09:23.517454+0000","last_fullsized":"2023-12-17T12:09:23.517454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T17:01:41.059044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514649+0000","last_change":"2023-12-17T12:08:49.193449+0000","last_active":"2023-12-17T12:09:23.514649+0000","last_peered":"2023-12-17T12:09:23.514649+0000","last_clean":"2023-12-17T12:09:23.514649+0000","last_became_active":"2023-12-17T12:08:49.192942+0000","last_became_peered":"2023-12-17T12:08:49.192942+0000","last_unstale":"2023-12-17T12:09:23.514649+0000","last_undegraded":"2023-12-17T12:09:23.514649+0000","last_fullsized":"2023-12-17T12:09:23.514649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:42:51.021808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514217+0000","last_change":"2023-12-17T12:08:49.193515+0000","last_active":"2023-12-17T12:09:23.514217+0000","last_peered":"2023-12-17T12:09:23.514217+0000","last_clean":"2023-12-17T12:09:23.514217+0000","last_became_active":"2023-12-17T12:08:49.193232+0000","last_became_peered":"2023-12-17T12:08:49.193232+0000","last_unstale":"2023-12-17T12:09:23.514217+0000","last_undegraded":"2023-12-17T12:09:23.514217+0000","last_fullsized":"2023-12-17T12:09:23.514217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:26:59.948350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.514611+0000","last_change":"2023-12-17T12:08:49.197110+0000","last_active":"2023-12-17T12:09:23.514611+0000","last_peered":"2023-12-17T12:09:23.514611+0000","last_clean":"2023-12-17T12:09:23.514611+0000","last_became_active":"2023-12-17T12:08:49.196712+0000","last_became_peered":"2023-12-17T12:08:49.196712+0000","last_unstale":"2023-12-17T12:09:23.514611+0000","last_undegraded":"2023-12-17T12:09:23.514611+0000","last_fullsized":"2023-12-17T12:09:23.514611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:21:40.555247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517792+0000","last_change":"2023-12-17T12:08:49.192021+0000","last_active":"2023-12-17T12:09:23.517792+0000","last_peered":"2023-12-17T12:09:23.517792+0000","last_clean":"2023-12-17T12:09:23.517792+0000","last_became_active":"2023-12-17T12:08:49.191696+0000","last_became_peered":"2023-12-17T12:08:49.191696+0000","last_unstale":"2023-12-17T12:09:23.517792+0000","last_undegraded":"2023-12-17T12:09:23.517792+0000","last_fullsized":"2023-12-17T12:09:23.517792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:27:40.425617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-17T12:09:25.050811+0000","last_change":"2023-12-17T12:09:25.050811+0000","last_active":"2023-12-17T12:09:25.050811+0000","last_peered":"2023-12-17T12:09:25.050811+0000","last_clean":"2023-12-17T12:09:25.050811+0000","last_became_active":"2023-12-17T12:09:25.050385+0000","last_became_peered":"2023-12-17T12:09:25.050385+0000","last_unstale":"2023-12-17T12:09:25.050811+0000","last_undegraded":"2023-12-17T12:09:25.050811+0000","last_fullsized":"2023-12-17T12:09:25.050811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:43:43.241960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515723+0000","last_change":"2023-12-17T12:08:49.191147+0000","last_active":"2023-12-17T12:09:23.515723+0000","last_peered":"2023-12-17T12:09:23.515723+0000","last_clean":"2023-12-17T12:09:23.515723+0000","last_became_active":"2023-12-17T12:08:49.190804+0000","last_became_peered":"2023-12-17T12:08:49.190804+0000","last_unstale":"2023-12-17T12:09:23.515723+0000","last_undegraded":"2023-12-17T12:09:23.515723+0000","last_fullsized":"2023-12-17T12:09:23.515723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:09:40.551342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515591+0000","last_change":"2023-12-17T12:08:49.184707+0000","last_active":"2023-12-17T12:09:23.515591+0000","last_peered":"2023-12-17T12:09:23.515591+0000","last_clean":"2023-12-17T12:09:23.515591+0000","last_became_active":"2023-12-17T12:08:49.183644+0000","last_became_peered":"2023-12-17T12:08:49.183644+0000","last_unstale":"2023-12-17T12:09:23.515591+0000","last_undegraded":"2023-12-17T12:09:23.515591+0000","last_fullsized":"2023-12-17T12:09:23.515591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T18:30:43.845054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.520483+0000","last_change":"2023-12-17T12:08:49.195299+0000","last_active":"2023-12-17T12:09:23.520483+0000","last_peered":"2023-12-17T12:09:23.520483+0000","last_clean":"2023-12-17T12:09:23.520483+0000","last_became_active":"2023-12-17T12:08:49.194315+0000","last_became_peered":"2023-12-17T12:08:49.194315+0000","last_unstale":"2023-12-17T12:09:23.520483+0000","last_undegraded":"2023-12-17T12:09:23.520483+0000","last_fullsized":"2023-12-17T12:09:23.520483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:33:26.056073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.517180+0000","last_change":"2023-12-17T12:08:49.196145+0000","last_active":"2023-12-17T12:09:23.517180+0000","last_peered":"2023-12-17T12:09:23.517180+0000","last_clean":"2023-12-17T12:09:23.517180+0000","last_became_active":"2023-12-17T12:08:49.195893+0000","last_became_peered":"2023-12-17T12:08:49.195893+0000","last_unstale":"2023-12-17T12:09:23.517180+0000","last_undegraded":"2023-12-17T12:09:23.517180+0000","last_fullsized":"2023-12-17T12:09:23.517180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:33:23.868301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.519244+0000","last_change":"2023-12-17T12:08:49.193113+0000","last_active":"2023-12-17T12:09:23.519244+0000","last_peered":"2023-12-17T12:09:23.519244+0000","last_clean":"2023-12-17T12:09:23.519244+0000","last_became_active":"2023-12-17T12:08:49.191730+0000","last_became_peered":"2023-12-17T12:08:49.191730+0000","last_unstale":"2023-12-17T12:09:23.519244+0000","last_undegraded":"2023-12-17T12:09:23.519244+0000","last_fullsized":"2023-12-17T12:09:23.519244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T12:45:33.858040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.511007+0000","last_change":"2023-12-17T12:08:49.193401+0000","last_active":"2023-12-17T12:09:23.511007+0000","last_peered":"2023-12-17T12:09:23.511007+0000","last_clean":"2023-12-17T12:09:23.511007+0000","last_became_active":"2023-12-17T12:08:49.192993+0000","last_became_peered":"2023-12-17T12:08:49.192993+0000","last_unstale":"2023-12-17T12:09:23.511007+0000","last_undegraded":"2023-12-17T12:09:23.511007+0000","last_fullsized":"2023-12-17T12:09:23.511007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T15:21:16.594494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.513097+0000","last_change":"2023-12-17T12:08:49.200282+0000","last_active":"2023-12-17T12:09:23.513097+0000","last_peered":"2023-12-17T12:09:23.513097+0000","last_clean":"2023-12-17T12:09:23.513097+0000","last_became_active":"2023-12-17T12:08:49.199788+0000","last_became_peered":"2023-12-17T12:08:49.199788+0000","last_unstale":"2023-12-17T12:09:23.513097+0000","last_undegraded":"2023-12-17T12:09:23.513097+0000","last_fullsized":"2023-12-17T12:09:23.513097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T21:07:19.164767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-17T12:09:23.515160+0000","last_change":"2023-12-17T12:08:49.185712+0000","last_active":"2023-12-17T12:09:23.515160+0000","last_peered":"2023-12-17T12:09:23.515160+0000","last_clean":"2023-12-17T12:09:23.515160+0000","last_became_active":"2023-12-17T12:08:49.185417+0000","last_became_peered":"2023-12-17T12:08:49.185417+0000","last_unstale":"2023-12-17T12:09:23.515160+0000","last_undegraded":"2023-12-17T12:09:23.515160+0000","last_fullsized":"2023-12-17T12:09:23.515160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-17T12:08:48.367447+0000","last_clean_scrub_stamp":"2023-12-17T12:08:48.367447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T14:28:43.076574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246978,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61803,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":261,"num_read_kb":193,"num_write":147,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":58386,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":185409,"data_compressed":9050,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481814,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":441806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.83999999999999997}]},{"osd":2,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.93100000000000005}]},{"osd":5,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.88400000000000001}]},{"osd":6,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.81599999999999995}]},{"osd":7,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.99099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":9,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":1.0469999999999999}]},{"osd":10,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.028},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.91600000000000004}]}]},{"osd":10,"up_from":69,"seq":296352743449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1091486,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.436}]},{"osd":2,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.61599999999999999}]},{"osd":3,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.503},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63400000000000001}]},{"osd":4,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.83099999999999996}]},{"osd":6,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.82599999999999996}]},{"osd":8,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.78700000000000003}]},{"osd":9,"last update":"Sun Dec 17 12:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58999999999999997}]},{"osd":11,"last update":"Sun Dec 17 12:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.88200000000000001}]}]},{"osd":9,"up_from":64,"seq":274877906974,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":442165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69399999999999995}]},{"osd":1,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.117,"15min":1.117},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.83699999999999997,"15min":0.85099999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.80500000000000005}]},{"osd":2,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.123,"5min":1.306,"15min":1.306},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":1.462,"15min":1.5780000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.097,"5min":13.166,"15min":13.166},"last":1}]},{"osd":3,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84199999999999997,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.85099999999999998,"15min":0.88100000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":1.917,"15min":1.917},"last":0.75600000000000001}]},{"osd":4,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.91900000000000004,"15min":0.94299999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0549999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93000000000000005,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":1.232,"15min":1.302},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.135,"5min":7.1920000000000002,"15min":7.1920000000000002},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.88700000000000001,"15min":0.90100000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":1.0189999999999999,"15min":1.0549999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.048,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.875,"15min":0.89700000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.0780000000000001,"15min":1.1100000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.208,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.876},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.1020000000000001,"15min":1.137},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.214,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.83499999999999996}]},{"osd":8,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.071,"15min":1.097},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3320000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.6679999999999999,"15min":1.7929999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.274,"5min":13.18,"15min":13.18},"last":1.0169999999999999}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":13.212,"5min":13.212,"15min":13.212},"last":1.0509999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":8,"up_from":56,"seq":240518168611,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":441380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85799999999999998,"5min":1.091,"15min":1.091},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.98499999999999999,"15min":1.0509999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79000000000000004,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.623}]},{"osd":1,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2050000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.748,"5min":1.0509999999999999,"15min":1.101},"min":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.135,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":0.55400000000000005}]},{"osd":2,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.68400000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.96699999999999997,"5min":1.048,"15min":1.048},"last":0.81599999999999995}]},{"osd":3,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":1.079,"15min":1.151},"min":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96299999999999997,"5min":7.0599999999999996,"15min":7.0599999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":1.1259999999999999,"15min":1.1950000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.097,"5min":6.992,"15min":6.992},"last":0.83799999999999997}]},{"osd":4,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":1.087,"15min":1.149},"min":{"1min":0.54200000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.179,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.95899999999999996}]},{"osd":5,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.79400000000000004,"15min":0.80400000000000005},"min":{"1min":0.46100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.0189999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.53300000000000003}]},{"osd":6,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.88700000000000001,"15min":0.89700000000000002},"min":{"1min":0.628,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.018},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.95099999999999996,"15min":0.96499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.276,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92900000000000005,"15min":0.93100000000000005},"min":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.66000000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.73699999999999999}]},{"osd":9,"last update":"Sun Dec 17 12:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.90500000000000003,"15min":0.89500000000000002},"min":{"1min":0.70399999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.94199999999999995,"15min":0.92800000000000005},"min":{"1min":0.80100000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.86699999999999999}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":1033565,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92200000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57599999999999996}]},{"osd":1,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.496,"15min":0.48099999999999998},"min":{"1min":0.38900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53000000000000003}]},{"osd":2,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.78900000000000003,"15min":0.745},"min":{"1min":0.52800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":7.1070000000000002,"5min":7.1070000000000002,"15min":7.1070000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69699999999999995,"15min":0.69499999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.83999999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95899999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65300000000000002}]},{"osd":4,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90500000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94999999999999996,"5min":1.119,"15min":1.119},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.84899999999999998,"15min":0.80100000000000005},"min":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":7.1669999999999998,"5min":7.1669999999999998,"15min":7.1669999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.79200000000000004}]},{"osd":6,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.84199999999999997,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0780000000000001,"5min":1.21,"15min":1.21},"last":0.63300000000000001}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.69799999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.143,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.125},"min":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.101,"5min":7.1369999999999996,"15min":7.1369999999999996},"last":0.76000000000000001}]},{"osd":9,"last update":"Sun Dec 17 12:09:22 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":0.96699999999999997,"15min":0.89900000000000002},"min":{"1min":0.68500000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":8.6199999999999992,"5min":8.6199999999999992,"15min":8.6199999999999992},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85299999999999998,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.90900000000000003}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.95699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]}]},{"osd":6,"up_from":44,"seq":188978561069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":441437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65200000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68500000000000005,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.754,"15min":0.754},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.66000000000000003}]},{"osd":2,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72999999999999998,"15min":0.72699999999999998},"min":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86599999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3540000000000001,"5min":5.6740000000000004,"15min":5.6740000000000004},"last":0.83899999999999997}]},{"osd":3,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.621},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":0.44}]},{"osd":4,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":1.1160000000000001,"5min":1.167,"15min":1.167},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81499999999999995,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.879,"15min":0.86899999999999999},"min":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.1880000000000002,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.75700000000000001}]},{"osd":7,"last update":"Sun Dec 17 12:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.749},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.623}]},{"osd":8,"last update":"Sun Dec 17 12:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.99399999999999999,"15min":1.008},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1970000000000001,"5min":1.419,"15min":1.419},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.95199999999999996,"15min":0.96599999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.214,"5min":1.298,"15min":1.298},"last":0.91900000000000004}]},{"osd":9,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.88500000000000001,"15min":0.88900000000000001},"min":{"1min":0.70699999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.236,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94999999999999996,"15min":0.96099999999999997},"min":{"1min":0.57599999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1399999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.68899999999999995}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.70999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":499610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.61799999999999999,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64300000000000002,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":1.032,"15min":1.032},"last":0.58699999999999997}]},{"osd":2,"last update":"Sun Dec 17 12:08:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63800000000000001,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0549999999999999,"5min":1.111,"15min":1.111},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63400000000000001,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.097,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.036}]},{"osd":3,"last update":"Sun Dec 17 12:09:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.65300000000000002,"15min":0.60199999999999998},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.65900000000000003,"15min":0.60199999999999998},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.119,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.91800000000000004}]},{"osd":4,"last update":"Sun Dec 17 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.50700000000000001},"min":{"1min":0.38300000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.94299999999999995,"5min":1.01,"15min":1.01},"last":0.81999999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.69699999999999995,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.83799999999999997}]},{"osd":6,"last update":"Sun Dec 17 12:09:22 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.75800000000000001,"15min":0.72599999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.77500000000000002,"15min":0.73999999999999999},"min":{"1min":0.54300000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1439999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Dec 17 12:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68899999999999995,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79000000000000004}]},{"osd":8,"last update":"Sun Dec 17 12:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.77400000000000002,"15min":0.75700000000000001},"min":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.153},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.84099999999999997,"15min":0.81999999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.127}]},{"osd":9,"last update":"Sun Dec 17 12:09:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.58399999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.879},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.879,"15min":0.871},"min":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.177,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.1859999999999999}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.94199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":442702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Dec 17 12:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.61399999999999999},"min":{"1min":0.442,"5min":0.372,"15min":0.372},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67900000000000005,"15min":0.63500000000000001},"min":{"1min":0.42899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Sun Dec 17 12:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65900000000000003,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.747},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65000000000000002,"15min":0.59999999999999998},"min":{"1min":0.55900000000000005,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.72799999999999998}]},{"osd":3,"last update":"Sun Dec 17 12:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.375,"5min":0.315,"15min":0.315},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.505},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46800000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69499999999999995}]},{"osd":4,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68700000000000006,"15min":0.64500000000000002},"min":{"1min":0.504,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.077,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:08:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78200000000000003,"15min":0.75},"min":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0680000000000001,"5min":1.248,"15min":1.248},"last":0.92000000000000004}]},{"osd":6,"last update":"Sun Dec 17 12:09:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.367,"5min":2.367,"15min":2.367},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70799999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.84299999999999997,"15min":0.83399999999999996},"min":{"1min":0.71999999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.871,"15min":0.86299999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.81599999999999995}]},{"osd":8,"last update":"Sun Dec 17 12:08:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85399999999999998,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.95899999999999996},"min":{"1min":0.753,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.1100000000000001,"5min":1.413,"15min":1.413},"last":0.84099999999999997}]},{"osd":9,"last update":"Sun Dec 17 12:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.27,"5min":1.321,"15min":1.321},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.88200000000000001,"15min":0.89500000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.76300000000000001}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.80100000000000005}]},{"osd":11,"last update":"Sun Dec 17 12:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.90000000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":442574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.52400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.48199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.626}]},{"osd":1,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.66800000000000004,"15min":0.621},"min":{"1min":0.54500000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98099999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.95599999999999996}]},{"osd":3,"last update":"Sun Dec 17 12:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90100000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":0.67100000000000004}]},{"osd":4,"last update":"Sun Dec 17 12:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67200000000000004,"15min":0.64700000000000002},"min":{"1min":0.442,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.69199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95799999999999996,"5min":1.181,"15min":1.181},"last":0.90200000000000002}]},{"osd":5,"last update":"Sun Dec 17 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94699999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Dec 17 12:09:17 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.79700000000000004,"15min":0.72899999999999998},"min":{"1min":0.73099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.016,"5min":1.214,"15min":1.214},"last":0.84199999999999997}]},{"osd":7,"last update":"Sun Dec 17 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.75900000000000001}]},{"osd":8,"last update":"Sun Dec 17 12:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80600000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.82699999999999996,"15min":0.82399999999999995},"min":{"1min":0.63,"5min":0.501,"15min":0.501},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0369999999999999}]},{"osd":9,"last update":"Sun Dec 17 12:09:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93500000000000005,"15min":0.93400000000000005},"min":{"1min":0.78600000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84399999999999997,"15min":0.83799999999999997},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.92200000000000004}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.1060000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":441631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.44900000000000001,"15min":0.40300000000000002},"min":{"1min":0.33700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.47599999999999998,"15min":0.42799999999999999},"min":{"1min":0.37,"5min":0.31,"15min":0.31},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70399999999999996}]},{"osd":1,"last update":"Sun Dec 17 12:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.69399999999999995,"15min":0.626},"min":{"1min":0.49399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.67300000000000004,"15min":0.623},"min":{"1min":0.54400000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91100000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.054}]},{"osd":2,"last update":"Sun Dec 17 12:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.623},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71799999999999997,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Sun Dec 17 12:08:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.67500000000000004,"15min":0.63500000000000001},"min":{"1min":0.623,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Sun Dec 17 12:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.014,"5min":1.028,"15min":1.028},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.73399999999999999},"min":{"1min":0.53300000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0820000000000001,"5min":1.278,"15min":1.278},"last":0.80100000000000005}]},{"osd":6,"last update":"Sun Dec 17 12:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93700000000000006,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.72199999999999998,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1279999999999999,"5min":1.429,"15min":1.429},"last":0.51900000000000002}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84999999999999998,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.92600000000000005}]},{"osd":8,"last update":"Sun Dec 17 12:07:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.71999999999999997}]},{"osd":9,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.65000000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.107,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.873},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.91000000000000003,"15min":0.91700000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.96499999999999997}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0029999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":500061,"data_compressed":9050,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.47399999999999998},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48599999999999999,"15min":0.44400000000000001},"min":{"1min":0.38900000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.621}]},{"osd":2,"last update":"Sun Dec 17 12:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76400000000000001,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95499999999999996,"5min":5.125,"15min":5.125},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68500000000000005,"15min":0.64600000000000002},"min":{"1min":0.56699999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95199999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.94299999999999995}]},{"osd":3,"last update":"Sun Dec 17 12:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66600000000000004,"15min":0.624},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96599999999999997,"5min":1.139,"15min":1.139},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67000000000000004,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96599999999999997,"5min":1.222,"15min":1.222},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Dec 17 12:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68400000000000005,"15min":0.65900000000000003},"min":{"1min":0.59599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68799999999999994,"15min":0.66800000000000004},"min":{"1min":0.57599999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90600000000000003,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0089999999999999}]},{"osd":6,"last update":"Sun Dec 17 12:09:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.89500000000000002,"15min":0.95299999999999996},"min":{"1min":0.59699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":0.997,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.57999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0649999999999999,"5min":1.278,"15min":1.278},"last":0.84999999999999998}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.70299999999999996,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.83899999999999997}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.036,"5min":1.339,"15min":1.339},"last":1.093},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.53600000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.03,"5min":1.325,"15min":1.325},"last":0.97499999999999998}]},{"osd":9,"last update":"Sun Dec 17 12:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.66200000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1240000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.86899999999999999,"15min":0.86199999999999999},"min":{"1min":0.74199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.056,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Sun Dec 17 12:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.93200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]}]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":1033191,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94999999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.51500000000000001}]},{"osd":1,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.66300000000000003,"15min":0.64300000000000002},"min":{"1min":0.55100000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89400000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66200000000000003,"15min":0.63500000000000001},"min":{"1min":0.53400000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77200000000000002}]},{"osd":2,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.876,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.39700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66400000000000003}]},{"osd":3,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95099999999999996,"5min":1.097,"15min":1.097},"last":0.72699999999999998}]},{"osd":4,"last update":"Sun Dec 17 12:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":0.92700000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.52600000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95499999999999996,"5min":1.048,"15min":1.048},"last":0.92200000000000004}]},{"osd":6,"last update":"Sun Dec 17 12:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78400000000000003,"15min":0.77100000000000002},"min":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0309999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.79900000000000004,"15min":0.78500000000000003},"min":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97899999999999998,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.75}]},{"osd":7,"last update":"Sun Dec 17 12:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":1.0589999999999999,"5min":1.117,"15min":1.117},"last":0.879},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":0.83899999999999997},"min":{"1min":0.63500000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.95899999999999996,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.86199999999999999}]},{"osd":8,"last update":"Sun Dec 17 12:08:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.78300000000000003,"15min":0.77900000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.105,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.83999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1419999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.78800000000000003}]},{"osd":9,"last update":"Sun Dec 17 12:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.79300000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.085,"5min":1.232,"15min":1.232},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.123,"5min":1.137,"15min":1.137},"last":0.82499999999999996}]},{"osd":10,"last update":"Sun Dec 17 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34109,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34109,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35752,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":58837,"data_compressed":9050,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-17T12:09:41.192 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-17T12:09:41.192 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-17T12:09:41.193 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-17T12:09:41.193 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- ceph health --format=json 2023-12-17T12:09:41.625 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:41.837 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:41 smithi027 ceph-mon[36871]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:42.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:41 smithi141 ceph-mon[41225]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:42.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:41 smithi192 ceph-mon[41899]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 -- 172.21.15.27:0/2509201069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 msgr2=0x7f1b80107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 --2- 172.21.15.27:0/2509201069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 0x7f1b80107ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009990 tx=0x7f1b7c009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 -- 172.21.15.27:0/2509201069 shutdown_connections 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 --2- 172.21.15.27:0/2509201069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b80102040 0x7f1b8010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 --2- 172.21.15.27:0/2509201069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 0x7f1b80107ad0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 --2- 172.21.15.27:0/2509201069 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1b80101680 0x7f1b80101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 -- 172.21.15.27:0/2509201069 >> 172.21.15.27:0/2509201069 conn(0x7f1b800787c0 msgr2=0x7f1b80078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 -- 172.21.15.27:0/2509201069 shutdown_connections 2023-12-17T12:09:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.859+0000 7f1b87d49700 1 -- 172.21.15.27:0/2509201069 wait complete. 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.860+0000 7f1b87d49700 1 Processor -- start 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.860+0000 7f1b87d49700 1 -- start start 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b87d49700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b80101680 0x7f1b8019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b87d49700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1b80102040 0x7f1b8019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b87d49700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 0x7f1b80197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b87d49700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80197e00 con 0x7f1b801076f0 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b87d49700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80197f70 con 0x7f1b80102040 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b87d49700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80198130 con 0x7f1b80101680 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b862e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 0x7f1b80197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b862e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 0x7f1b80197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43682/0 (socket says 172.21.15.27:43682) 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b862e6700 1 -- 172.21.15.27:0/4090160813 learned_addr learned my addr 172.21.15.27:0/4090160813 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.861+0000 7f1b85ae5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b80101680 0x7f1b8019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b852e4700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1b80102040 0x7f1b8019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b862e6700 1 -- 172.21.15.27:0/4090160813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b80101680 msgr2=0x7f1b8019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b862e6700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b80101680 0x7f1b8019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b862e6700 1 -- 172.21.15.27:0/4090160813 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1b80102040 msgr2=0x7f1b8019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b862e6700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1b80102040 0x7f1b8019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b862e6700 1 -- 172.21.15.27:0/4090160813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c0095f0 con 0x7f1b801076f0 2023-12-17T12:09:42.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b862e6700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 0x7f1b80197790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800efc0 tx=0x7f1b7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b76ffd700 1 -- 172.21.15.27:0/4090160813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78018070 con 0x7f1b801076f0 2023-12-17T12:09:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b76ffd700 1 -- 172.21.15.27:0/4090160813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b7800f040 con 0x7f1b801076f0 2023-12-17T12:09:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b801983b0 con 0x7f1b801076f0 2023-12-17T12:09:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b76ffd700 1 -- 172.21.15.27:0/4090160813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800cbb0 con 0x7f1b801076f0 2023-12-17T12:09:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.862+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b801a4640 con 0x7f1b801076f0 2023-12-17T12:09:42.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.863+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b80109b60 con 0x7f1b801076f0 2023-12-17T12:09:42.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.865+0000 7f1b76ffd700 1 -- 172.21.15.27:0/4090160813 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1b7800cd90 con 0x7f1b801076f0 2023-12-17T12:09:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.867+0000 7f1b76ffd700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1b6c0a1590 0x7f1b6c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.867+0000 7f1b76ffd700 1 -- 172.21.15.27:0/4090160813 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1b780cc810 con 0x7f1b801076f0 2023-12-17T12:09:42.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.867+0000 7f1b85ae5700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1b6c0a1590 0x7f1b6c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:42.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.868+0000 7f1b85ae5700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1b6c0a1590 0x7f1b6c0a3a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006010 tx=0x7f1b7c00b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:42.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:42.871+0000 7f1b76ffd700 1 -- 172.21.15.27:0/4090160813 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1b78095d50 con 0x7f1b801076f0 2023-12-17T12:09:42.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:42 smithi027 ceph-mon[36871]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 4.7 KiB/s, 0 objects/s recovering 2023-12-17T12:09:43.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:42 smithi141 ceph-mon[41225]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 4.7 KiB/s, 0 objects/s recovering 2023-12-17T12:09:43.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:42 smithi192 ceph-mon[41899]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 4.7 KiB/s, 0 objects/s recovering 2023-12-17T12:09:43.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.127+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1b80102d10 con 0x7f1b801076f0 2023-12-17T12:09:43.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.128+0000 7f1b76ffd700 1 -- 172.21.15.27:0/4090160813 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1b78099370 con 0x7f1b801076f0 2023-12-17T12:09:43.128 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:43.128 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-17T12:09:43.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1b6c0a1590 msgr2=0x7f1b6c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:43.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1b6c0a1590 0x7f1b6c0a3a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006010 tx=0x7f1b7c00b260 comp rx=0 tx=0).stop 2023-12-17T12:09:43.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 msgr2=0x7f1b80197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 0x7f1b80197790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800efc0 tx=0x7f1b7800c3a0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 shutdown_connections 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f1b6c0a1590 0x7f1b6c0a3a50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b80101680 0x7f1b8019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b801076f0 0x7f1b80197790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 --2- 172.21.15.27:0/4090160813 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1b80102040 0x7f1b8019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 >> 172.21.15.27:0/4090160813 conn(0x7f1b800787c0 msgr2=0x7f1b801004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 shutdown_connections 2023-12-17T12:09:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.130+0000 7f1b87d49700 1 -- 172.21.15.27:0/4090160813 wait complete. 2023-12-17T12:09:43.594 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-12-17T12:09:43.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/1202685640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb84102690 msgr2=0x7fcb84102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/1202685640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb84102690 0x7fcb84102af0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb780099c0 tx=0x7fcb78009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/1202685640 shutdown_connections 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/1202685640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb841014a0 0x7fcb841018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/1202685640 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcb84103030 0x7fcb841098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/1202685640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb84102690 0x7fcb84102af0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/1202685640 >> 172.21.15.27:0/1202685640 conn(0x7fcb840fca10 msgr2=0x7fcb840fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/1202685640 shutdown_connections 2023-12-17T12:09:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.807+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/1202685640 wait complete. 2023-12-17T12:09:43.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.809+0000 7fcb8a7ff700 1 Processor -- start 2023-12-17T12:09:43.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.809+0000 7fcb8a7ff700 1 -- start start 2023-12-17T12:09:43.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.809+0000 7fcb8a7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb841014a0 0x7fcb84197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:43.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.809+0000 7fcb8a7ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcb84102690 0x7fcb841978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:43.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.809+0000 7fcb8a7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb84103030 0x7fcb8419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:43.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.809+0000 7fcb8a7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8419ee70 con 0x7fcb841014a0 2023-12-17T12:09:43.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.809+0000 7fcb8a7ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8419efb0 con 0x7fcb84102690 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.809+0000 7fcb8a7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8419f140 con 0x7fcb84103030 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb841014a0 0x7fcb84197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb841014a0 0x7fcb84197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43690/0 (socket says 172.21.15.27:43690) 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 -- 172.21.15.27:0/2051891679 learned_addr learned my addr 172.21.15.27:0/2051891679 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb88d9c700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb84103030 0x7fcb8419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb837fe700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcb84102690 0x7fcb841978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 -- 172.21.15.27:0/2051891679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb84103030 msgr2=0x7fcb8419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb84103030 0x7fcb8419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 -- 172.21.15.27:0/2051891679 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcb84102690 msgr2=0x7fcb841978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcb84102690 0x7fcb841978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 -- 172.21.15.27:0/2051891679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780095f0 con 0x7fcb841014a0 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb837fe700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcb84102690 0x7fcb841978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb88d9c700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb84103030 0x7fcb8419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.810+0000 7fcb83fff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb841014a0 0x7fcb84197390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400d980 tx=0x7fcb7400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.811+0000 7fcb817fa700 1 -- 172.21.15.27:0/2051891679 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400de60 con 0x7fcb841014a0 2023-12-17T12:09:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.811+0000 7fcb817fa700 1 -- 172.21.15.27:0/2051891679 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb740044b0 con 0x7fcb841014a0 2023-12-17T12:09:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.811+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8419f420 con 0x7fcb841014a0 2023-12-17T12:09:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.811+0000 7fcb817fa700 1 -- 172.21.15.27:0/2051891679 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74003b40 con 0x7fcb841014a0 2023-12-17T12:09:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.811+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8419f930 con 0x7fcb841014a0 2023-12-17T12:09:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.812+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb84108990 con 0x7fcb841014a0 2023-12-17T12:09:43.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.814+0000 7fcb817fa700 1 -- 172.21.15.27:0/2051891679 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fcb74003ca0 con 0x7fcb841014a0 2023-12-17T12:09:43.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.815+0000 7fcb817fa700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcb6c0a1510 0x7fcb6c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:43.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.815+0000 7fcb817fa700 1 -- 172.21.15.27:0/2051891679 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcb740ccbf0 con 0x7fcb841014a0 2023-12-17T12:09:43.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.816+0000 7fcb837fe700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcb6c0a1510 0x7fcb6c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:43.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.816+0000 7fcb837fe700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcb6c0a1510 0x7fcb6c0a39d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb78006010 tx=0x7fcb78005c00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:43.819+0000 7fcb817fa700 1 -- 172.21.15.27:0/2051891679 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcb740960a0 con 0x7fcb841014a0 2023-12-17T12:09:43.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:43 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4090160813' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-17T12:09:44.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:43 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4090160813' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-17T12:09:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.035+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb8410cbc0 con 0x7fcb841014a0 2023-12-17T12:09:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.037+0000 7fcb817fa700 1 -- 172.21.15.27:0/2051891679 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22507 (secure 0 0 0) 0x7fcb74009940 con 0x7fcb841014a0 2023-12-17T12:09:44.037 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:44.037 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:09:24.507787+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcb6c0a1510 msgr2=0x7fcb6c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcb6c0a1510 0x7fcb6c0a39d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb78006010 tx=0x7fcb78005c00 comp rx=0 tx=0).stop 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb841014a0 msgr2=0x7fcb84197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb841014a0 0x7fcb84197390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400d980 tx=0x7fcb7400dd40 comp rx=0 tx=0).stop 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 shutdown_connections 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcb6c0a1510 0x7fcb6c0a39d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb84103030 0x7fcb8419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb841014a0 0x7fcb84197390 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.039+0000 7fcb8a7ff700 1 --2- 172.21.15.27:0/2051891679 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcb84102690 0x7fcb841978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.040+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 >> 172.21.15.27:0/2051891679 conn(0x7fcb840fca10 msgr2=0x7fcb84105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.040+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 shutdown_connections 2023-12-17T12:09:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.040+0000 7fcb8a7ff700 1 -- 172.21.15.27:0/2051891679 wait complete. 2023-12-17T12:09:44.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-17T12:04:41.530587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-17T12:08:47.025470+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-17T12:08:48.366216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-17T12:09:44.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-17T12:09:44.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:43 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4090160813' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-17T12:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 -- 172.21.15.27:0/1654325992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec075610 msgr2=0x7f9eec073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/1654325992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec075610 0x7f9eec073a60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009990 tx=0x7f9edc009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 -- 172.21.15.27:0/1654325992 shutdown_connections 2023-12-17T12:09:44.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/1654325992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eec109460 0x7f9eec10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/1654325992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec075610 0x7f9eec073a60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/1654325992 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9eec074030 0x7f9eec0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 -- 172.21.15.27:0/1654325992 >> 172.21.15.27:0/1654325992 conn(0x7f9eec0fcb10 msgr2=0x7f9eec0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 -- 172.21.15.27:0/1654325992 shutdown_connections 2023-12-17T12:09:44.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.271+0000 7f9ef21ff700 1 -- 172.21.15.27:0/1654325992 wait complete. 2023-12-17T12:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.272+0000 7f9ef21ff700 1 Processor -- start 2023-12-17T12:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.272+0000 7f9ef21ff700 1 -- start start 2023-12-17T12:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.272+0000 7f9ef21ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9eec074030 0x7f9eec199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.272+0000 7f9ef21ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eec075610 0x7f9eec199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.272+0000 7f9ef21ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec109460 0x7f9eec1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.272+0000 7f9ef21ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec1a1170 con 0x7f9eec109460 2023-12-17T12:09:44.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.272+0000 7f9ef21ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec1a12b0 con 0x7f9eec074030 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9ef21ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec1a1440 con 0x7f9eec075610 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9eec074030 0x7f9eec199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9eec074030 0x7f9eec199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:39042/0 (socket says 172.21.15.27:39042) 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 -- 172.21.15.27:0/4105002418 learned_addr learned my addr 172.21.15.27:0/4105002418 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eebfff700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec109460 0x7f9eec1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeaffd700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eec075610 0x7f9eec199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 -- 172.21.15.27:0/4105002418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eec075610 msgr2=0x7f9eec199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eec075610 0x7f9eec199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 -- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec109460 msgr2=0x7f9eec1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec109460 0x7f9eec1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 -- 172.21.15.27:0/4105002418 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0095f0 con 0x7f9eec074030 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.273+0000 7f9eeb7fe700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9eec074030 0x7f9eec199640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9edc000c00 tx=0x7f9edc0047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:44.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.274+0000 7f9ee8ff9700 1 -- 172.21.15.27:0/4105002418 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc01d070 con 0x7f9eec074030 2023-12-17T12:09:44.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.274+0000 7f9ee8ff9700 1 -- 172.21.15.27:0/4105002418 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9edc004100 con 0x7f9eec074030 2023-12-17T12:09:44.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.274+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec1a16c0 con 0x7f9eec074030 2023-12-17T12:09:44.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.274+0000 7f9ee8ff9700 1 -- 172.21.15.27:0/4105002418 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc022dd0 con 0x7f9eec074030 2023-12-17T12:09:44.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.274+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec1a1b40 con 0x7f9eec074030 2023-12-17T12:09:44.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.275+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec108220 con 0x7f9eec074030 2023-12-17T12:09:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.278+0000 7f9ee8ff9700 1 -- 172.21.15.27:0/4105002418 <== mon.2 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9edc022570 con 0x7f9eec074030 2023-12-17T12:09:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.279+0000 7f9ee8ff9700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9ed40a1590 0x7f9ed40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.279+0000 7f9ee8ff9700 1 -- 172.21.15.27:0/4105002418 <== mon.2 v2:172.21.15.141:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9edc0ce760 con 0x7f9eec074030 2023-12-17T12:09:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.279+0000 7f9eeaffd700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9ed40a1590 0x7f9ed40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.280+0000 7f9eeaffd700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9ed40a1590 0x7f9ed40a3a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9eec075330 tx=0x7f9ee000a3b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:44.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.281+0000 7f9ee8ff9700 1 -- 172.21.15.27:0/4105002418 <== mon.2 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9edc097ca0 con 0x7f9eec074030 2023-12-17T12:09:44.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.495+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9eec10f0a0 con 0x7f9eec074030 2023-12-17T12:09:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.496+0000 7f9ee8ff9700 1 -- 172.21.15.27:0/4105002418 <== mon.2 v2:172.21.15.141:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9edc027020 con 0x7f9eec074030 2023-12-17T12:09:44.496 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9ed40a1590 msgr2=0x7f9ed40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9ed40a1590 0x7f9ed40a3a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9eec075330 tx=0x7f9ee000a3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9eec074030 msgr2=0x7f9eec199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9eec074030 0x7f9eec199640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9edc000c00 tx=0x7f9edc0047a0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 shutdown_connections 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9ed40a1590 0x7f9ed40a3a50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec109460 0x7f9eec1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9eec074030 0x7f9eec199640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 --2- 172.21.15.27:0/4105002418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eec075610 0x7f9eec199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 >> 172.21.15.27:0/4105002418 conn(0x7f9eec0fcb10 msgr2=0x7f9eec105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.498+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 shutdown_connections 2023-12-17T12:09:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.499+0000 7f9ef21ff700 1 -- 172.21.15.27:0/4105002418 wait complete. 2023-12-17T12:09:44.518 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-17T12:09:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.728+0000 7fec3d59d700 1 -- 172.21.15.27:0/701379213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec38102f70 msgr2=0x7fec38109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.728+0000 7fec3d59d700 1 --2- 172.21.15.27:0/701379213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec38102f70 0x7fec38109820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec3000b0d0 tx=0x7fec3000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.729+0000 7fec3d59d700 1 -- 172.21.15.27:0/701379213 shutdown_connections 2023-12-17T12:09:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.729+0000 7fec3d59d700 1 --2- 172.21.15.27:0/701379213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec381013e0 0x7fec38101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.729+0000 7fec3d59d700 1 --2- 172.21.15.27:0/701379213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec38102f70 0x7fec38109820 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.729+0000 7fec3d59d700 1 --2- 172.21.15.27:0/701379213 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec381025d0 0x7fec38102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.729+0000 7fec3d59d700 1 -- 172.21.15.27:0/701379213 >> 172.21.15.27:0/701379213 conn(0x7fec380fc930 msgr2=0x7fec380fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.729+0000 7fec3d59d700 1 -- 172.21.15.27:0/701379213 shutdown_connections 2023-12-17T12:09:44.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.729+0000 7fec3d59d700 1 -- 172.21.15.27:0/701379213 wait complete. 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec3d59d700 1 Processor -- start 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec3d59d700 1 -- start start 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec3d59d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec381013e0 0x7fec38199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec3d59d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec381025d0 0x7fec38199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec3d59d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec38102f70 0x7fec381a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec3d59d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381a1060 con 0x7fec381025d0 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec3d59d700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381a11a0 con 0x7fec38102f70 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec3d59d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381a1330 con 0x7fec381013e0 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec381025d0 0x7fec38199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec381025d0 0x7fec38199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43710/0 (socket says 172.21.15.27:43710) 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec377fe700 1 -- 172.21.15.27:0/3620940702 learned_addr learned my addr 172.21.15.27:0/3620940702 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.730+0000 7fec37fff700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec381013e0 0x7fec38199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec3cd9c700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec38102f70 0x7fec381a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec377fe700 1 -- 172.21.15.27:0/3620940702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec381013e0 msgr2=0x7fec38199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec377fe700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec381013e0 0x7fec38199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec377fe700 1 -- 172.21.15.27:0/3620940702 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec38102f70 msgr2=0x7fec381a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec377fe700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec38102f70 0x7fec381a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec377fe700 1 -- 172.21.15.27:0/3620940702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec30009d00 con 0x7fec381025d0 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec3cd9c700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec38102f70 0x7fec381a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec377fe700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec381025d0 0x7fec38199ac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00c9b0 tx=0x7fec2c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec357fa700 1 -- 172.21.15.27:0/3620940702 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c007b00 con 0x7fec381025d0 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec357fa700 1 -- 172.21.15.27:0/3620940702 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec2c007c60 con 0x7fec381025d0 2023-12-17T12:09:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec357fa700 1 -- 172.21.15.27:0/3620940702 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c018660 con 0x7fec381025d0 2023-12-17T12:09:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.731+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec381a1610 con 0x7fec381025d0 2023-12-17T12:09:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.732+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec381a1ad0 con 0x7fec381025d0 2023-12-17T12:09:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.733+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec380681c0 con 0x7fec381025d0 2023-12-17T12:09:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.735+0000 7fec357fa700 1 -- 172.21.15.27:0/3620940702 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fec2c00f450 con 0x7fec381025d0 2023-12-17T12:09:44.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.736+0000 7fec357fa700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec200a1590 0x7fec200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:44.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.736+0000 7fec357fa700 1 -- 172.21.15.27:0/3620940702 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fec2c0ccd70 con 0x7fec381025d0 2023-12-17T12:09:44.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.736+0000 7fec37fff700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec200a1590 0x7fec200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:44.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.737+0000 7fec37fff700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec200a1590 0x7fec200a3a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec38102430 tx=0x7fec28007400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:44.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.740+0000 7fec357fa700 1 -- 172.21.15.27:0/3620940702 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fec2c096280 con 0x7fec381025d0 2023-12-17T12:09:44.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:44 smithi027 ceph-mon[36871]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:44.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:44 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2051891679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:44.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:44 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4105002418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:09:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.949+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fec3810cd60 con 0x7fec381025d0 2023-12-17T12:09:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.950+0000 7fec357fa700 1 -- 172.21.15.27:0/3620940702 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fec2c00fe60 con 0x7fec381025d0 2023-12-17T12:09:44.950 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-17T12:09:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec200a1590 msgr2=0x7fec200a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec200a1590 0x7fec200a3a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec38102430 tx=0x7fec28007400 comp rx=0 tx=0).stop 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec381025d0 msgr2=0x7fec38199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec381025d0 0x7fec38199ac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00c9b0 tx=0x7fec2c00ccc0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 shutdown_connections 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec200a1590 0x7fec200a3a50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec381013e0 0x7fec38199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec38102f70 0x7fec381a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 --2- 172.21.15.27:0/3620940702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec381025d0 0x7fec38199ac0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 >> 172.21.15.27:0/3620940702 conn(0x7fec380fc930 msgr2=0x7fec38073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.952+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 shutdown_connections 2023-12-17T12:09:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:44.953+0000 7fec3d59d700 1 -- 172.21.15.27:0/3620940702 wait complete. 2023-12-17T12:09:44.972 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-17T12:09:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:44 smithi141 ceph-mon[41225]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:44 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2051891679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:45.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:44 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4105002418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:09:45.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:44 smithi192 ceph-mon[41899]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:45.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:44 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2051891679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:45.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:44 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4105002418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:09:45.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.182+0000 7f90da7ff700 1 -- 172.21.15.27:0/4290204891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4107260 msgr2=0x7f90d41096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.182+0000 7f90da7ff700 1 --2- 172.21.15.27:0/4290204891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4107260 0x7f90d41096d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00b0a0 tx=0x7f90cc00b3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.183+0000 7f90da7ff700 1 -- 172.21.15.27:0/4290204891 shutdown_connections 2023-12-17T12:09:45.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.183+0000 7f90da7ff700 1 --2- 172.21.15.27:0/4290204891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4107260 0x7f90d41096d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.183+0000 7f90da7ff700 1 --2- 172.21.15.27:0/4290204891 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f90d4102000 0x7f90d41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.183+0000 7f90da7ff700 1 --2- 172.21.15.27:0/4290204891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d4104930 0x7f90d4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.183+0000 7f90da7ff700 1 -- 172.21.15.27:0/4290204891 >> 172.21.15.27:0/4290204891 conn(0x7f90d40fb9f0 msgr2=0x7f90d40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:45.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.183+0000 7f90da7ff700 1 -- 172.21.15.27:0/4290204891 shutdown_connections 2023-12-17T12:09:45.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.183+0000 7f90da7ff700 1 -- 172.21.15.27:0/4290204891 wait complete. 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.184+0000 7f90da7ff700 1 Processor -- start 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.184+0000 7f90da7ff700 1 -- start start 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.184+0000 7f90da7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4102000 0x7f90d4197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.184+0000 7f90da7ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f90d4104930 0x7f90d4197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.184+0000 7f90da7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d4107260 0x7f90d419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.184+0000 7f90da7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d419ef70 con 0x7f90d4102000 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.184+0000 7f90da7ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d419f0b0 con 0x7f90d4104930 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.184+0000 7f90da7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d419f240 con 0x7f90d4107260 2023-12-17T12:09:45.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4102000 0x7f90d4197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d37fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f90d4104930 0x7f90d4197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4102000 0x7f90d4197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43732/0 (socket says 172.21.15.27:43732) 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 -- 172.21.15.27:0/1936858091 learned_addr learned my addr 172.21.15.27:0/1936858091 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d8d9c700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d4107260 0x7f90d419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 -- 172.21.15.27:0/1936858091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d4107260 msgr2=0x7f90d419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d4107260 0x7f90d419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 -- 172.21.15.27:0/1936858091 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f90d4104930 msgr2=0x7f90d4197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f90d4104930 0x7f90d4197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 -- 172.21.15.27:0/1936858091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc009d00 con 0x7f90d4102000 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d37fe700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f90d4104930 0x7f90d4197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.185+0000 7f90d3fff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4102000 0x7f90d4197440 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e9a0 tx=0x7f90c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.186+0000 7f90d17fa700 1 -- 172.21.15.27:0/1936858091 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4009790 con 0x7f90d4102000 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.186+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d419f520 con 0x7f90d4102000 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.186+0000 7f90d17fa700 1 -- 172.21.15.27:0/1936858091 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90c400cbd0 con 0x7f90d4102000 2023-12-17T12:09:45.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.186+0000 7f90d17fa700 1 -- 172.21.15.27:0/1936858091 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4018680 con 0x7f90d4102000 2023-12-17T12:09:45.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.186+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d419fa80 con 0x7f90d4102000 2023-12-17T12:09:45.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.187+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d40fd570 con 0x7f90d4102000 2023-12-17T12:09:45.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.189+0000 7f90d17fa700 1 -- 172.21.15.27:0/1936858091 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f90c400cd40 con 0x7f90d4102000 2023-12-17T12:09:45.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.190+0000 7f90d17fa700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f90bc0a1590 0x7f90bc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:45.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.190+0000 7f90d17fa700 1 -- 172.21.15.27:0/1936858091 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f90c4014070 con 0x7f90d4102000 2023-12-17T12:09:45.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.191+0000 7f90d37fe700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f90bc0a1590 0x7f90bc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:45.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.191+0000 7f90d37fe700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f90bc0a1590 0x7f90bc0a3a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f90c8005fd0 tx=0x7f90c8005e30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:45.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.195+0000 7f90d17fa700 1 -- 172.21.15.27:0/1936858091 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f90c40522f0 con 0x7f90d4102000 2023-12-17T12:09:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.409+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90d40682e0 con 0x7f90d4102000 2023-12-17T12:09:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.409+0000 7f90d17fa700 1 -- 172.21.15.27:0/1936858091 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90c40965a0 con 0x7f90d4102000 2023-12-17T12:09:45.409 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-17T12:09:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f90bc0a1590 msgr2=0x7f90bc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f90bc0a1590 0x7f90bc0a3a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f90c8005fd0 tx=0x7f90c8005e30 comp rx=0 tx=0).stop 2023-12-17T12:09:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4102000 msgr2=0x7f90d4197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4102000 0x7f90d4197440 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e9a0 tx=0x7f90c400ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 shutdown_connections 2023-12-17T12:09:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f90bc0a1590 0x7f90bc0a3a50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d4107260 0x7f90d419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4102000 0x7f90d4197440 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 --2- 172.21.15.27:0/1936858091 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f90d4104930 0x7f90d4197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 >> 172.21.15.27:0/1936858091 conn(0x7f90d40fb9f0 msgr2=0x7f90d4102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 shutdown_connections 2023-12-17T12:09:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.412+0000 7f90da7ff700 1 -- 172.21.15.27:0/1936858091 wait complete. 2023-12-17T12:09:45.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:09:45.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3117448795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 msgr2=0x7f7f641042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3117448795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 0x7f7f641042f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009990 tx=0x7f7f54009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3117448795 shutdown_connections 2023-12-17T12:09:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3117448795 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7f64107160 0x7f7f641095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3117448795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 0x7f7f641042f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3117448795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f64104830 0x7f7f64106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3117448795 >> 172.21.15.27:0/3117448795 conn(0x7f7f640fb8f0 msgr2=0x7f7f640fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3117448795 shutdown_connections 2023-12-17T12:09:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.641+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3117448795 wait complete. 2023-12-17T12:09:45.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.642+0000 7f7f6a5ff700 1 Processor -- start 2023-12-17T12:09:45.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.642+0000 7f7f6a5ff700 1 -- start start 2023-12-17T12:09:45.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f6a5ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 0x7f7f64197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f6a5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f64104830 0x7f7f64197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f6a5ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7f64107160 0x7f7f6419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f6a5ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6419ee60 con 0x7f7f64101f00 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f6a5ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6419efa0 con 0x7f7f64107160 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f6a5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6419f130 con 0x7f7f64104830 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f63fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 0x7f7f64197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f63fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 0x7f7f64197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43758/0 (socket says 172.21.15.27:43758) 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f63fff700 1 -- 172.21.15.27:0/3947962407 learned_addr learned my addr 172.21.15.27:0/3947962407 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f637fe700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f64104830 0x7f7f64197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f63fff700 1 -- 172.21.15.27:0/3947962407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f64104830 msgr2=0x7f7f64197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f63fff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f64104830 0x7f7f64197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f63fff700 1 -- 172.21.15.27:0/3947962407 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7f64107160 msgr2=0x7f7f6419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f63fff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7f64107160 0x7f7f6419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.643+0000 7f7f63fff700 1 -- 172.21.15.27:0/3947962407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540095f0 con 0x7f7f64101f00 2023-12-17T12:09:45.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.644+0000 7f7f63fff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 0x7f7f64197330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7f54003ec0 tx=0x7f7f540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.644+0000 7f7f617fa700 1 -- 172.21.15.27:0/3947962407 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5401d070 con 0x7f7f64101f00 2023-12-17T12:09:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.644+0000 7f7f617fa700 1 -- 172.21.15.27:0/3947962407 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f5400bdb0 con 0x7f7f64101f00 2023-12-17T12:09:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.644+0000 7f7f617fa700 1 -- 172.21.15.27:0/3947962407 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400fe50 con 0x7f7f64101f00 2023-12-17T12:09:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.644+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6419f3b0 con 0x7f7f64101f00 2023-12-17T12:09:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.644+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6419f780 con 0x7f7f64101f00 2023-12-17T12:09:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.645+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f640fd470 con 0x7f7f64101f00 2023-12-17T12:09:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.647+0000 7f7f617fa700 1 -- 172.21.15.27:0/3947962407 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7f5400f5f0 con 0x7f7f64101f00 2023-12-17T12:09:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.648+0000 7f7f617fa700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7f4c0a1590 0x7f7f4c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.649+0000 7f7f617fa700 1 -- 172.21.15.27:0/3947962407 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7f540ce4d0 con 0x7f7f64101f00 2023-12-17T12:09:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.649+0000 7f7f637fe700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7f4c0a1590 0x7f7f4c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.649+0000 7f7f637fe700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7f4c0a1590 0x7f7f4c0a3a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009bf0 tx=0x7f7f58009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:45.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.650+0000 7f7f617fa700 1 -- 172.21.15.27:0/3947962407 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7f54097a70 con 0x7f7f64101f00 2023-12-17T12:09:45.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:45 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3620940702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-17T12:09:45.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:45 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1936858091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-17T12:09:45.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.915+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f64068230 con 0x7f7f64101f00 2023-12-17T12:09:45.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.915+0000 7f7f617fa700 1 -- 172.21.15.27:0/3947962407 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11327 (secure 0 0 0) 0x7f7f5409b090 con 0x7f7f64101f00 2023-12-17T12:09:45.916 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:45.917 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3551652066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3551652066},{"type":"v1","addr":"172.21.15.27:6841","nonce":3551652066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-17T12:08:48.917098+0000","modified":"2023-12-17T12:08:55.420903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24538,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.141:6839/3282544216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6838","nonce":3282544216},{"type":"v1","addr":"172.21.15.141:6839","nonce":3282544216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-17T12:09:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7f4c0a1590 msgr2=0x7f7f4c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7f4c0a1590 0x7f7f4c0a3a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009bf0 tx=0x7f7f58009260 comp rx=0 tx=0).stop 2023-12-17T12:09:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 msgr2=0x7f7f64197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 0x7f7f64197330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7f54003ec0 tx=0x7f7f540057f0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 shutdown_connections 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f7f4c0a1590 0x7f7f4c0a3a50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7f64107160 0x7f7f6419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101f00 0x7f7f64197330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 --2- 172.21.15.27:0/3947962407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f64104830 0x7f7f64197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 >> 172.21.15.27:0/3947962407 conn(0x7f7f640fb8f0 msgr2=0x7f7f64103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 shutdown_connections 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:45.918+0000 7f7f6a5ff700 1 -- 172.21.15.27:0/3947962407 wait complete. 2023-12-17T12:09:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-17T12:09:45.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:09:46.013 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:45 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3620940702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-17T12:09:46.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:45 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1936858091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-17T12:09:46.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:45 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3620940702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-17T12:09:46.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:45 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1936858091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 -- 172.21.15.27:0/3837163936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 msgr2=0x7f4bb0102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/3837163936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 0x7f4bb0102af0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00099c0 tx=0x7f4ba0009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 -- 172.21.15.27:0/3837163936 shutdown_connections 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/3837163936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb0103030 0x7f4bb01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/3837163936 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4bb01014a0 0x7f4bb01018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/3837163936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 0x7f4bb0102af0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 -- 172.21.15.27:0/3837163936 >> 172.21.15.27:0/3837163936 conn(0x7f4bb00fca10 msgr2=0x7f4bb00fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 -- 172.21.15.27:0/3837163936 shutdown_connections 2023-12-17T12:09:46.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.145+0000 7f4bb87ff700 1 -- 172.21.15.27:0/3837163936 wait complete. 2023-12-17T12:09:46.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.146+0000 7f4bb87ff700 1 Processor -- start 2023-12-17T12:09:46.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.146+0000 7f4bb87ff700 1 -- start start 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.146+0000 7f4bb87ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb01014a0 0x7f4bb01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.146+0000 7f4bb87ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 0x7f4bb01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.146+0000 7f4bb87ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4bb0103030 0x7f4bb019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.146+0000 7f4bb87ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb019ede0 con 0x7f4bb0102690 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.146+0000 7f4bb87ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb019ef20 con 0x7f4bb0103030 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.146+0000 7f4bb87ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb019f0b0 con 0x7f4bb01014a0 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 0x7f4bb01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 0x7f4bb01977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43424/0 (socket says 172.21.15.27:43424) 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 -- 172.21.15.27:0/4004591682 learned_addr learned my addr 172.21.15.27:0/4004591682 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb659b700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb01014a0 0x7f4bb01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb6d9c700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4bb0103030 0x7f4bb019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:46.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 -- 172.21.15.27:0/4004591682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb01014a0 msgr2=0x7f4bb01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb01014a0 0x7f4bb01972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 -- 172.21.15.27:0/4004591682 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4bb0103030 msgr2=0x7f4bb019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4bb0103030 0x7f4bb019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 -- 172.21.15.27:0/4004591682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba00095f0 con 0x7f4bb0102690 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.147+0000 7f4bb5d9a700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 0x7f4bb01977f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009990 tx=0x7f4ba000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.148+0000 7f4ba77fe700 1 -- 172.21.15.27:0/4004591682 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba001d070 con 0x7f4bb0102690 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.148+0000 7f4ba77fe700 1 -- 172.21.15.27:0/4004591682 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ba000bd10 con 0x7f4bb0102690 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.148+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb019f330 con 0x7f4bb0102690 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.148+0000 7f4ba77fe700 1 -- 172.21.15.27:0/4004591682 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0017650 con 0x7f4bb0102690 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.148+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb019f750 con 0x7f4bb0102690 2023-12-17T12:09:46.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.149+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb01088c0 con 0x7f4bb0102690 2023-12-17T12:09:46.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.151+0000 7f4ba77fe700 1 -- 172.21.15.27:0/4004591682 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f4ba0022470 con 0x7f4bb0102690 2023-12-17T12:09:46.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.153+0000 7f4ba77fe700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b9c0a1510 0x7f4b9c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:46.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.153+0000 7f4ba77fe700 1 -- 172.21.15.27:0/4004591682 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ba00ce140 con 0x7f4bb0102690 2023-12-17T12:09:46.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.153+0000 7f4bb659b700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b9c0a1510 0x7f4b9c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:46.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.154+0000 7f4bb659b700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b9c0a1510 0x7f4b9c0a39d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005fd0 tx=0x7f4bac005e50 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:46.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.157+0000 7f4ba77fe700 1 -- 172.21.15.27:0/4004591682 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4ba0097680 con 0x7f4bb0102690 2023-12-17T12:09:46.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.372+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bb010cbc0 con 0x7f4bb0102690 2023-12-17T12:09:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.373+0000 7f4ba77fe700 1 -- 172.21.15.27:0/4004591682 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22507 (secure 0 0 0) 0x7f4ba009aca0 con 0x7f4bb0102690 2023-12-17T12:09:46.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:46.375 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:09:24.507787+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:09:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b9c0a1510 msgr2=0x7f4b9c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b9c0a1510 0x7f4b9c0a39d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005fd0 tx=0x7f4bac005e50 comp rx=0 tx=0).stop 2023-12-17T12:09:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 msgr2=0x7f4bb01977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 0x7f4bb01977f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009990 tx=0x7f4ba000fab0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 shutdown_connections 2023-12-17T12:09:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b9c0a1510 0x7f4b9c0a39d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb01014a0 0x7f4bb01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4bb0103030 0x7f4bb019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 --2- 172.21.15.27:0/4004591682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bb0102690 0x7f4bb01977f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 >> 172.21.15.27:0/4004591682 conn(0x7f4bb00fca10 msgr2=0x7f4bb0105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 shutdown_connections 2023-12-17T12:09:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.376+0000 7f4bb87ff700 1 -- 172.21.15.27:0/4004591682 wait complete. 2023-12-17T12:09:46.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-17T12:04:41.530587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-17T12:08:47.025470+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-17T12:08:48.366216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-17T12:09:46.397 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-17T12:09:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 -- 172.21.15.27:0/765799814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 msgr2=0x7f4b3c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 --2- 172.21.15.27:0/765799814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 0x7f4b3c102bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4b300099c0 tx=0x7f4b30009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 -- 172.21.15.27:0/765799814 shutdown_connections 2023-12-17T12:09:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 --2- 172.21.15.27:0/765799814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b3c103130 0x7f4b3c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 --2- 172.21.15.27:0/765799814 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4b3c1015a0 0x7f4b3c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 --2- 172.21.15.27:0/765799814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 0x7f4b3c102bf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 -- 172.21.15.27:0/765799814 >> 172.21.15.27:0/765799814 conn(0x7f4b3c0fcb10 msgr2=0x7f4b3c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:46.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 -- 172.21.15.27:0/765799814 shutdown_connections 2023-12-17T12:09:46.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.608+0000 7f4b411ff700 1 -- 172.21.15.27:0/765799814 wait complete. 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.609+0000 7f4b411ff700 1 Processor -- start 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.609+0000 7f4b411ff700 1 -- start start 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.609+0000 7f4b411ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4b3c1015a0 0x7f4b3c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.609+0000 7f4b411ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 0x7f4b3c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.609+0000 7f4b411ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b3c103130 0x7f4b3c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.609+0000 7f4b411ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c1a11e0 con 0x7f4b3c102790 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.609+0000 7f4b411ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c1a1320 con 0x7f4b3c1015a0 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.609+0000 7f4b411ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c1a14b0 con 0x7f4b3c103130 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 0x7f4b3c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 0x7f4b3c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43452/0 (socket says 172.21.15.27:43452) 2023-12-17T12:09:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 -- 172.21.15.27:0/3518251021 learned_addr learned my addr 172.21.15.27:0/3518251021 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3ad9d700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4b3c1015a0 0x7f4b3c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3b59e700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b3c103130 0x7f4b3c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 -- 172.21.15.27:0/3518251021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b3c103130 msgr2=0x7f4b3c1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b3c103130 0x7f4b3c1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 -- 172.21.15.27:0/3518251021 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4b3c1015a0 msgr2=0x7f4b3c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4b3c1015a0 0x7f4b3c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 -- 172.21.15.27:0/3518251021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300095f0 con 0x7f4b3c102790 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3b59e700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b3c103130 0x7f4b3c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3ad9d700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4b3c1015a0 0x7f4b3c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.610+0000 7f4b3a59c700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 0x7f4b3c199c40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b30005910 tx=0x7f4b30005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.611+0000 7f4b23fff700 1 -- 172.21.15.27:0/3518251021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3001d070 con 0x7f4b3c102790 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.611+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c1a1730 con 0x7f4b3c102790 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.611+0000 7f4b23fff700 1 -- 172.21.15.27:0/3518251021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b3000bd30 con 0x7f4b3c102790 2023-12-17T12:09:46.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.611+0000 7f4b23fff700 1 -- 172.21.15.27:0/3518251021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30022dc0 con 0x7f4b3c102790 2023-12-17T12:09:46.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.611+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c1a1bb0 con 0x7f4b3c102790 2023-12-17T12:09:46.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.612+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c068310 con 0x7f4b3c102790 2023-12-17T12:09:46.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.614+0000 7f4b23fff700 1 -- 172.21.15.27:0/3518251021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f4b30004010 con 0x7f4b3c102790 2023-12-17T12:09:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.615+0000 7f4b23fff700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b240a1590 0x7f4b240a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.615+0000 7f4b23fff700 1 -- 172.21.15.27:0/3518251021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4b300cdfd0 con 0x7f4b3c102790 2023-12-17T12:09:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.615+0000 7f4b3ad9d700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b240a1590 0x7f4b240a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:46.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.616+0000 7f4b3ad9d700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b240a1590 0x7f4b240a3a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005950 tx=0x7f4b2c0058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:46.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.617+0000 7f4b23fff700 1 -- 172.21.15.27:0/3518251021 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4b30097690 con 0x7f4b3c102790 2023-12-17T12:09:46.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:46 smithi027 ceph-mon[36871]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:09:46.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:46 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3947962407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:46.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:46 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4004591682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:46.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.830+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b3c1a1e70 con 0x7f4b3c102790 2023-12-17T12:09:46.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.831+0000 7f4b23fff700 1 -- 172.21.15.27:0/3518251021 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b3009acb0 con 0x7f4b3c102790 2023-12-17T12:09:46.832 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b240a1590 msgr2=0x7f4b240a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b240a1590 0x7f4b240a3a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005950 tx=0x7f4b2c0058e0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 msgr2=0x7f4b3c199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 0x7f4b3c199c40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b30005910 tx=0x7f4b30005b00 comp rx=0 tx=0).stop 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 shutdown_connections 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4b240a1590 0x7f4b240a3a50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b3c103130 0x7f4b3c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4b3c1015a0 0x7f4b3c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 --2- 172.21.15.27:0/3518251021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b3c102790 0x7f4b3c199c40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.833+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 >> 172.21.15.27:0/3518251021 conn(0x7f4b3c0fcb10 msgr2=0x7f4b3c107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.834+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 shutdown_connections 2023-12-17T12:09:46.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:46.834+0000 7f4b411ff700 1 -- 172.21.15.27:0/3518251021 wait complete. 2023-12-17T12:09:46.854 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-17T12:09:47.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:46 smithi141 ceph-mon[41225]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:09:47.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:46 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3947962407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:47.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:46 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4004591682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:47.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:46 smithi192 ceph-mon[41899]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:09:47.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:46 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3947962407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:47.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:46 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4004591682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 -- 172.21.15.27:0/1163805348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8103130 msgr2=0x7feea81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/1163805348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8103130 0x7feea81099e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feea000b0d0 tx=0x7feea000b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 -- 172.21.15.27:0/1163805348 shutdown_connections 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/1163805348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8103130 0x7feea81099e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/1163805348 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7feea81015a0 0x7feea81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/1163805348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea8102790 0x7feea8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 -- 172.21.15.27:0/1163805348 >> 172.21.15.27:0/1163805348 conn(0x7feea80fcb10 msgr2=0x7feea80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 -- 172.21.15.27:0/1163805348 shutdown_connections 2023-12-17T12:09:47.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.064+0000 7feeaf9ff700 1 -- 172.21.15.27:0/1163805348 wait complete. 2023-12-17T12:09:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.065+0000 7feeaf9ff700 1 Processor -- start 2023-12-17T12:09:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.065+0000 7feeaf9ff700 1 -- start start 2023-12-17T12:09:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeaf9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea81015a0 0x7feea8075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeaf9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8102790 0x7feea80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeaf9ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7feea8103130 0x7feea8073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeaf9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea80746a0 con 0x7feea8102790 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeaf9ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea8074810 con 0x7feea8103130 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeaf9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea81a1590 con 0x7feea81015a0 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeacf9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8102790 0x7feea80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeacf9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8102790 0x7feea80739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43474/0 (socket says 172.21.15.27:43474) 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeacf9a700 1 -- 172.21.15.27:0/2988845304 learned_addr learned my addr 172.21.15.27:0/2988845304 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeadf9c700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7feea8103130 0x7feea8073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feead79b700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea81015a0 0x7feea8075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeacf9a700 1 -- 172.21.15.27:0/2988845304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea81015a0 msgr2=0x7feea8075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeacf9a700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea81015a0 0x7feea8075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeacf9a700 1 -- 172.21.15.27:0/2988845304 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7feea8103130 msgr2=0x7feea8073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeacf9a700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7feea8103130 0x7feea8073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feeacf9a700 1 -- 172.21.15.27:0/2988845304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea0009d00 con 0x7feea8102790 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.066+0000 7feead79b700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea81015a0 0x7feea8075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.067+0000 7feeacf9a700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8102790 0x7feea80739c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fee9800e9a0 tx=0x7fee9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.067+0000 7fee9e7fc700 1 -- 172.21.15.27:0/2988845304 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98009790 con 0x7feea8102790 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.067+0000 7fee9e7fc700 1 -- 172.21.15.27:0/2988845304 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee98010430 con 0x7feea8102790 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.067+0000 7fee9e7fc700 1 -- 172.21.15.27:0/2988845304 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98018680 con 0x7feea8102790 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.067+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea81a1820 con 0x7feea8102790 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.067+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea81a1d30 con 0x7feea8102790 2023-12-17T12:09:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.068+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea8108a20 con 0x7feea8102790 2023-12-17T12:09:47.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.070+0000 7fee9e7fc700 1 -- 172.21.15.27:0/2988845304 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fee980105a0 con 0x7feea8102790 2023-12-17T12:09:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.071+0000 7fee9e7fc700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee940a1590 0x7fee940a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.072+0000 7fee9e7fc700 1 -- 172.21.15.27:0/2988845304 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fee98014070 con 0x7feea8102790 2023-12-17T12:09:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.072+0000 7feead79b700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee940a1590 0x7fee940a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.072+0000 7feead79b700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee940a1590 0x7fee940a3a50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feea4005950 tx=0x7feea40058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.075+0000 7fee9e7fc700 1 -- 172.21.15.27:0/2988845304 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fee98096520 con 0x7feea8102790 2023-12-17T12:09:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.288+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feea810cc70 con 0x7feea8102790 2023-12-17T12:09:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.289+0000 7fee9e7fc700 1 -- 172.21.15.27:0/2988845304 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee98099b40 con 0x7feea8102790 2023-12-17T12:09:47.290 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-17T12:09:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee940a1590 msgr2=0x7fee940a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee940a1590 0x7fee940a3a50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feea4005950 tx=0x7feea40058e0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8102790 msgr2=0x7feea80739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8102790 0x7feea80739c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fee9800e9a0 tx=0x7fee9800ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 shutdown_connections 2023-12-17T12:09:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fee940a1590 0x7fee940a3a50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea81015a0 0x7feea8075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7feea8103130 0x7feea8073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 --2- 172.21.15.27:0/2988845304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8102790 0x7feea80739c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 >> 172.21.15.27:0/2988845304 conn(0x7feea80fcb10 msgr2=0x7feea8105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:47.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 shutdown_connections 2023-12-17T12:09:47.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.291+0000 7feeaf9ff700 1 -- 172.21.15.27:0/2988845304 wait complete. 2023-12-17T12:09:47.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-17T12:09:47.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 -- 172.21.15.27:0/586058247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 msgr2=0x7f9b941099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 --2- 172.21.15.27:0/586058247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 0x7f9b941099e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b0d0 tx=0x7f9b8c00b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 -- 172.21.15.27:0/586058247 shutdown_connections 2023-12-17T12:09:47.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 --2- 172.21.15.27:0/586058247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b941015a0 0x7f9b941019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 --2- 172.21.15.27:0/586058247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 0x7f9b941099e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 --2- 172.21.15.27:0/586058247 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9b94102790 0x7f9b94102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 -- 172.21.15.27:0/586058247 >> 172.21.15.27:0/586058247 conn(0x7f9b940fcb10 msgr2=0x7f9b940fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:47.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 -- 172.21.15.27:0/586058247 shutdown_connections 2023-12-17T12:09:47.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.523+0000 7f9b99bff700 1 -- 172.21.15.27:0/586058247 wait complete. 2023-12-17T12:09:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.524+0000 7f9b99bff700 1 Processor -- start 2023-12-17T12:09:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.524+0000 7f9b99bff700 1 -- start start 2023-12-17T12:09:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b99bff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9b941015a0 0x7f9b94199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b99bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b94102790 0x7f9b94199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b99bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 0x7f9b941a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b99bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941a12e0 con 0x7f9b94103130 2023-12-17T12:09:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b99bff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941a1420 con 0x7f9b941015a0 2023-12-17T12:09:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b99bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941a15b0 con 0x7f9b94102790 2023-12-17T12:09:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b93fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 0x7f9b941a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b93fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 0x7f9b941a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43502/0 (socket says 172.21.15.27:43502) 2023-12-17T12:09:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b93fff700 1 -- 172.21.15.27:0/324858947 learned_addr learned my addr 172.21.15.27:0/324858947 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b93fff700 1 -- 172.21.15.27:0/324858947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b94102790 msgr2=0x7f9b94199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b93fff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b94102790 0x7f9b94199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b93fff700 1 -- 172.21.15.27:0/324858947 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9b941015a0 msgr2=0x7f9b94199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b93fff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9b941015a0 0x7f9b94199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.525+0000 7f9b93fff700 1 -- 172.21.15.27:0/324858947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c009d00 con 0x7f9b94103130 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.526+0000 7f9b93fff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 0x7f9b941a0d10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c007b60 tx=0x7f9b8c0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.526+0000 7f9b917fa700 1 -- 172.21.15.27:0/324858947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00e040 con 0x7f9b94103130 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.526+0000 7f9b917fa700 1 -- 172.21.15.27:0/324858947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b8c012e10 con 0x7f9b94103130 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.526+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b941a1830 con 0x7f9b94103130 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.526+0000 7f9b917fa700 1 -- 172.21.15.27:0/324858947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c01b450 con 0x7f9b94103130 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.526+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b941a1d00 con 0x7f9b94103130 2023-12-17T12:09:47.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.527+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b94068310 con 0x7f9b94103130 2023-12-17T12:09:47.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.529+0000 7f9b917fa700 1 -- 172.21.15.27:0/324858947 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9b8c012980 con 0x7f9b94103130 2023-12-17T12:09:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.530+0000 7f9b917fa700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9b800a1590 0x7f9b800a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.531+0000 7f9b917fa700 1 -- 172.21.15.27:0/324858947 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9b8c0cec80 con 0x7f9b94103130 2023-12-17T12:09:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.531+0000 7f9b937fe700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9b800a1590 0x7f9b800a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.531+0000 7f9b937fe700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9b800a1590 0x7f9b800a3a50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b941025f0 tx=0x7f9b84009470 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.535+0000 7f9b917fa700 1 -- 172.21.15.27:0/324858947 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9b8c098220 con 0x7f9b94103130 2023-12-17T12:09:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.747+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b941a20b0 con 0x7f9b94103130 2023-12-17T12:09:47.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.748+0000 7f9b917fa700 1 -- 172.21.15.27:0/324858947 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9b8c09b840 con 0x7f9b94103130 2023-12-17T12:09:47.749 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-17T12:09:47.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9b800a1590 msgr2=0x7f9b800a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9b800a1590 0x7f9b800a3a50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b941025f0 tx=0x7f9b84009470 comp rx=0 tx=0).stop 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 msgr2=0x7f9b941a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 0x7f9b941a0d10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c007b60 tx=0x7f9b8c0094f0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 shutdown_connections 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f9b800a1590 0x7f9b800a3a50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b94103130 0x7f9b941a0d10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9b941015a0 0x7f9b94199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 --2- 172.21.15.27:0/324858947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b94102790 0x7f9b94199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 >> 172.21.15.27:0/324858947 conn(0x7f9b940fcb10 msgr2=0x7f9b94073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 shutdown_connections 2023-12-17T12:09:47.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.750+0000 7f9b99bff700 1 -- 172.21.15.27:0/324858947 wait complete. 2023-12-17T12:09:47.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:09:47.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:47 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3518251021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:09:47.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:47 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2988845304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-17T12:09:47.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 -- 172.21.15.27:0/3560947611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384107160 msgr2=0x7f93841095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 --2- 172.21.15.27:0/3560947611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384107160 0x7f93841095d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f937c00b0a0 tx=0x7f937c00b3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 -- 172.21.15.27:0/3560947611 shutdown_connections 2023-12-17T12:09:47.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 --2- 172.21.15.27:0/3560947611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9384101f00 0x7f93841042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 --2- 172.21.15.27:0/3560947611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384107160 0x7f93841095d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 --2- 172.21.15.27:0/3560947611 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9384104830 0x7f9384106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 -- 172.21.15.27:0/3560947611 >> 172.21.15.27:0/3560947611 conn(0x7f93840fb8f0 msgr2=0x7f93840fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:47.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 -- 172.21.15.27:0/3560947611 shutdown_connections 2023-12-17T12:09:47.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.977+0000 7f938b9ff700 1 -- 172.21.15.27:0/3560947611 wait complete. 2023-12-17T12:09:47.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.979+0000 7f938b9ff700 1 Processor -- start 2023-12-17T12:09:47.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.979+0000 7f938b9ff700 1 -- start start 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.979+0000 7f938b9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384101f00 0x7f9384101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.979+0000 7f938b9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9384104830 0x7f93840ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.979+0000 7f938b9ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9384107160 0x7f93841003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.979+0000 7f938b9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384100b90 con 0x7f9384101f00 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.979+0000 7f938b9ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384100d00 con 0x7f9384107160 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.979+0000 7f938b9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93841a1430 con 0x7f9384104830 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384101f00 0x7f9384101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384101f00 0x7f9384101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43518/0 (socket says 172.21.15.27:43518) 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 -- 172.21.15.27:0/1374500335 learned_addr learned my addr 172.21.15.27:0/1374500335 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f9389f9c700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9384107160 0x7f93841003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f9388f9a700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9384104830 0x7f93840ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 -- 172.21.15.27:0/1374500335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9384104830 msgr2=0x7f93840ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9384104830 0x7f93840ffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 -- 172.21.15.27:0/1374500335 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9384107160 msgr2=0x7f93841003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9384107160 0x7f93841003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 -- 172.21.15.27:0/1374500335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c009d00 con 0x7f9384101f00 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938979b700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384101f00 0x7f9384101820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f938000d950 tx=0x7f938000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f937a7fc700 1 -- 172.21.15.27:0/1374500335 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93800098e0 con 0x7f9384101f00 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f937a7fc700 1 -- 172.21.15.27:0/1374500335 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9380010460 con 0x7f9384101f00 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f937a7fc700 1 -- 172.21.15.27:0/1374500335 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938000f620 con 0x7f9384101f00 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.980+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93841a16c0 con 0x7f9384101f00 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.981+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93841a1bd0 con 0x7f9384101f00 2023-12-17T12:09:47.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.982+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93840fd470 con 0x7f9384101f00 2023-12-17T12:09:47.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.984+0000 7f937a7fc700 1 -- 172.21.15.27:0/1374500335 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f93800105d0 con 0x7f9384101f00 2023-12-17T12:09:47.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.985+0000 7f937a7fc700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f93700a1590 0x7f93700a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:47.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.985+0000 7f937a7fc700 1 -- 172.21.15.27:0/1374500335 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f93800ccc20 con 0x7f9384101f00 2023-12-17T12:09:47.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.985+0000 7f9388f9a700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f93700a1590 0x7f93700a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:47.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.986+0000 7f9388f9a700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f93700a1590 0x7f93700a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9374009bf0 tx=0x7f9374009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:47.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:47.989+0000 7f937a7fc700 1 -- 172.21.15.27:0/1374500335 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f93800960d0 con 0x7f9384101f00 2023-12-17T12:09:48.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:47 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3518251021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:09:48.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:47 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2988845304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-17T12:09:48.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:47 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3518251021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-17T12:09:48.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:47 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2988845304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-17T12:09:48.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.251+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f938410ce40 con 0x7f9384101f00 2023-12-17T12:09:48.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.252+0000 7f937a7fc700 1 -- 172.21.15.27:0/1374500335 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11327 (secure 0 0 0) 0x7f9380016090 con 0x7f9384101f00 2023-12-17T12:09:48.253 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:48.253 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3551652066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3551652066},{"type":"v1","addr":"172.21.15.27:6841","nonce":3551652066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-17T12:08:48.917098+0000","modified":"2023-12-17T12:08:55.420903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24538,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.141:6839/3282544216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6838","nonce":3282544216},{"type":"v1","addr":"172.21.15.141:6839","nonce":3282544216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.254+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f93700a1590 msgr2=0x7f93700a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.254+0000 7f938b9ff700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f93700a1590 0x7f93700a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9374009bf0 tx=0x7f9374009260 comp rx=0 tx=0).stop 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.254+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384101f00 msgr2=0x7f9384101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.254+0000 7f938b9ff700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384101f00 0x7f9384101820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f938000d950 tx=0x7f938000dc60 comp rx=0 tx=0).stop 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.254+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 shutdown_connections 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.255+0000 7f938b9ff700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f93700a1590 0x7f93700a3a50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.255+0000 7f938b9ff700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9384107160 0x7f93841003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.255+0000 7f938b9ff700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9384101f00 0x7f9384101820 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.255+0000 7f938b9ff700 1 --2- 172.21.15.27:0/1374500335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9384104830 0x7f93840ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.255+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 >> 172.21.15.27:0/1374500335 conn(0x7f93840fb8f0 msgr2=0x7f9384102920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.255+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 shutdown_connections 2023-12-17T12:09:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.255+0000 7f938b9ff700 1 -- 172.21.15.27:0/1374500335 wait complete. 2023-12-17T12:09:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-17T12:09:48.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:09:48.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 -- 172.21.15.27:0/3572756255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c41014a0 msgr2=0x7f34c41018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/3572756255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c41014a0 0x7f34c41018c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34c00099c0 tx=0x7f34c0009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 -- 172.21.15.27:0/3572756255 shutdown_connections 2023-12-17T12:09:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/3572756255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34c4103030 0x7f34c41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/3572756255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c41014a0 0x7f34c41018c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/3572756255 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f34c4102690 0x7f34c4102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 -- 172.21.15.27:0/3572756255 >> 172.21.15.27:0/3572756255 conn(0x7f34c40fca10 msgr2=0x7f34c40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 -- 172.21.15.27:0/3572756255 shutdown_connections 2023-12-17T12:09:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.481+0000 7f34cc1ff700 1 -- 172.21.15.27:0/3572756255 wait complete. 2023-12-17T12:09:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.482+0000 7f34cc1ff700 1 Processor -- start 2023-12-17T12:09:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.482+0000 7f34cc1ff700 1 -- start start 2023-12-17T12:09:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.482+0000 7f34cc1ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f34c41014a0 0x7f34c4199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34cc1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34c4102690 0x7f34c41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34cc1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c4103030 0x7f34c41a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34cc1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c41a0fc0 con 0x7f34c4103030 2023-12-17T12:09:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34cc1ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c41a1100 con 0x7f34c41014a0 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34cc1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c41a1290 con 0x7f34c4102690 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34ca79c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c4103030 0x7f34c41a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34ca79c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c4103030 0x7f34c41a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43544/0 (socket says 172.21.15.27:43544) 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34ca79c700 1 -- 172.21.15.27:0/4019953570 learned_addr learned my addr 172.21.15.27:0/4019953570 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34c979a700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34c4102690 0x7f34c41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34c9f9b700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f34c41014a0 0x7f34c4199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34ca79c700 1 -- 172.21.15.27:0/4019953570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34c4102690 msgr2=0x7f34c41999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34ca79c700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34c4102690 0x7f34c41999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34ca79c700 1 -- 172.21.15.27:0/4019953570 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f34c41014a0 msgr2=0x7f34c4199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34ca79c700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f34c41014a0 0x7f34c4199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.483+0000 7f34ca79c700 1 -- 172.21.15.27:0/4019953570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c00095f0 con 0x7f34c4103030 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.484+0000 7f34ca79c700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c4103030 0x7f34c41a09f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34bc007d70 tx=0x7f34bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.484+0000 7f34baffd700 1 -- 172.21.15.27:0/4019953570 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc00ad40 con 0x7f34c4103030 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.484+0000 7f34baffd700 1 -- 172.21.15.27:0/4019953570 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34bc0128b0 con 0x7f34c4103030 2023-12-17T12:09:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.484+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c41a1570 con 0x7f34c4103030 2023-12-17T12:09:48.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.484+0000 7f34baffd700 1 -- 172.21.15.27:0/4019953570 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0115a0 con 0x7f34c4103030 2023-12-17T12:09:48.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.484+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c41a1a30 con 0x7f34c4103030 2023-12-17T12:09:48.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.485+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c4068230 con 0x7f34c4103030 2023-12-17T12:09:48.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.487+0000 7f34baffd700 1 -- 172.21.15.27:0/4019953570 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f34bc012a20 con 0x7f34c4103030 2023-12-17T12:09:48.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.488+0000 7f34baffd700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34b00a1510 0x7f34b00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:48.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.489+0000 7f34baffd700 1 -- 172.21.15.27:0/4019953570 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34bc0cd000 con 0x7f34c4103030 2023-12-17T12:09:48.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.489+0000 7f34c9f9b700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34b00a1510 0x7f34b00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:48.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.489+0000 7f34c9f9b700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34b00a1510 0x7f34b00a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f34c0000c00 tx=0x7f34c0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:48.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.492+0000 7f34baffd700 1 -- 172.21.15.27:0/4019953570 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f34bc1002f0 con 0x7f34c4103030 2023-12-17T12:09:48.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.710+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34c410cbb0 con 0x7f34c4103030 2023-12-17T12:09:48.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.712+0000 7f34baffd700 1 -- 172.21.15.27:0/4019953570 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22507 (secure 0 0 0) 0x7f34bc09a0b0 con 0x7f34c4103030 2023-12-17T12:09:48.713 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:48.714 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:09:24.507787+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:09:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.714+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34b00a1510 msgr2=0x7f34b00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.714+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34b00a1510 0x7f34b00a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f34c0000c00 tx=0x7f34c0005fb0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.714+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c4103030 msgr2=0x7f34c41a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.714+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c4103030 0x7f34c41a09f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34bc007d70 tx=0x7f34bc00a340 comp rx=0 tx=0).stop 2023-12-17T12:09:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.715+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 shutdown_connections 2023-12-17T12:09:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.715+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f34b00a1510 0x7f34b00a39d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.715+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c4103030 0x7f34c41a09f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.715+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f34c41014a0 0x7f34c4199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.715+0000 7f34cc1ff700 1 --2- 172.21.15.27:0/4019953570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34c4102690 0x7f34c41999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.715+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 >> 172.21.15.27:0/4019953570 conn(0x7f34c40fca10 msgr2=0x7f34c4073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.715+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 shutdown_connections 2023-12-17T12:09:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.715+0000 7f34cc1ff700 1 -- 172.21.15.27:0/4019953570 wait complete. 2023-12-17T12:09:48.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:09:48.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:48 smithi027 ceph-mon[36871]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:09:48.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:48 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/324858947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-17T12:09:48.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:48 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1374500335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:48.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.942+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1102833039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b4101090 msgr2=0x7f51b4101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.942+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1102833039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b4101090 0x7f51b4101510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00b0a0 tx=0x7f51ac00b3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.942+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1102833039 shutdown_connections 2023-12-17T12:09:48.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.942+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1102833039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b4101090 0x7f51b4101510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.942+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1102833039 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51b40ffd70 0x7f51b4100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.942+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1102833039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51b41006d0 0x7f51b4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.942+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1102833039 >> 172.21.15.27:0/1102833039 conn(0x7f51b40fb8f0 msgr2=0x7f51b40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:48.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.942+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1102833039 shutdown_connections 2023-12-17T12:09:48.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.943+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1102833039 wait complete. 2023-12-17T12:09:48.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51bc7ff700 1 Processor -- start 2023-12-17T12:09:48.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51bc7ff700 1 -- start start 2023-12-17T12:09:48.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51bc7ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51b40ffd70 0x7f51b4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:48.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51bc7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b41006d0 0x7f51b4199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:48.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51bc7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51b4101090 0x7f51b41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51bc7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41a1160 con 0x7f51b41006d0 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51bc7ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41a12a0 con 0x7f51b40ffd70 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51bc7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41a1430 con 0x7f51b4101090 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51b9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b41006d0 0x7f51b4199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51b9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b41006d0 0x7f51b4199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43572/0 (socket says 172.21.15.27:43572) 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51b9d9a700 1 -- 172.21.15.27:0/1061086026 learned_addr learned my addr 172.21.15.27:0/1061086026 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.944+0000 7f51ba59b700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51b40ffd70 0x7f51b4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51bad9c700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51b4101090 0x7f51b41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51b9d9a700 1 -- 172.21.15.27:0/1061086026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51b4101090 msgr2=0x7f51b41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51b9d9a700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51b4101090 0x7f51b41a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51b9d9a700 1 -- 172.21.15.27:0/1061086026 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51b40ffd70 msgr2=0x7f51b4199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51b9d9a700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51b40ffd70 0x7f51b4199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:48.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51b9d9a700 1 -- 172.21.15.27:0/1061086026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac009d00 con 0x7f51b41006d0 2023-12-17T12:09:48.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51b9d9a700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b41006d0 0x7f51b4199b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f51a400e9a0 tx=0x7f51a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:48.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51ab7fe700 1 -- 172.21.15.27:0/1061086026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4009790 con 0x7f51b41006d0 2023-12-17T12:09:48.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51ab7fe700 1 -- 172.21.15.27:0/1061086026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51a4010430 con 0x7f51b41006d0 2023-12-17T12:09:48.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51ab7fe700 1 -- 172.21.15.27:0/1061086026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a400cea0 con 0x7f51b41006d0 2023-12-17T12:09:48.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b41a1710 con 0x7f51b41006d0 2023-12-17T12:09:48.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.945+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b41a1bd0 con 0x7f51b41006d0 2023-12-17T12:09:48.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.947+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b410ade0 con 0x7f51b41006d0 2023-12-17T12:09:48.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.949+0000 7f51ab7fe700 1 -- 172.21.15.27:0/1061086026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f51a40105a0 con 0x7f51b41006d0 2023-12-17T12:09:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.950+0000 7f51ab7fe700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51a00a1590 0x7f51a00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.950+0000 7f51ab7fe700 1 -- 172.21.15.27:0/1061086026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f51a4014070 con 0x7f51b41006d0 2023-12-17T12:09:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.950+0000 7f51ba59b700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51a00a1590 0x7f51a00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:48.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.951+0000 7f51ba59b700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51a00a1590 0x7f51a00a3a50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f51b0007c60 tx=0x7f51b00073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:48.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:48.954+0000 7f51ab7fe700 1 -- 172.21.15.27:0/1061086026 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f51a4096430 con 0x7f51b41006d0 2023-12-17T12:09:49.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:48 smithi141 ceph-mon[41225]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:09:49.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:48 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/324858947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-17T12:09:49.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:48 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1374500335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:49.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:48 smithi192 ceph-mon[41899]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:09:49.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:48 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/324858947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-17T12:09:49.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:48 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1374500335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:49.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.212+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51b4068230 con 0x7f51b41006d0 2023-12-17T12:09:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.213+0000 7f51ab7fe700 1 -- 172.21.15.27:0/1061086026 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11327 (secure 0 0 0) 0x7f51a4099a50 con 0x7f51b41006d0 2023-12-17T12:09:49.214 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:09:49.215 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3551652066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3551652066},{"type":"v1","addr":"172.21.15.27:6841","nonce":3551652066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-17T12:08:48.917098+0000","modified":"2023-12-17T12:08:55.420903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24538,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.141:6839/3282544216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6838","nonce":3282544216},{"type":"v1","addr":"172.21.15.141:6839","nonce":3282544216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-17T12:09:49.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51a00a1590 msgr2=0x7f51a00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51a00a1590 0x7f51a00a3a50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f51b0007c60 tx=0x7f51b00073d0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b41006d0 msgr2=0x7f51b4199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b41006d0 0x7f51b4199b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f51a400e9a0 tx=0x7f51a400ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 shutdown_connections 2023-12-17T12:09:49.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f51a00a1590 0x7f51a00a3a50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51b4101090 0x7f51b41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f51b40ffd70 0x7f51b4199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 --2- 172.21.15.27:0/1061086026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b41006d0 0x7f51b4199b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.215+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 >> 172.21.15.27:0/1061086026 conn(0x7f51b40fb8f0 msgr2=0x7f51b40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.216+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 shutdown_connections 2023-12-17T12:09:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.216+0000 7f51bc7ff700 1 -- 172.21.15.27:0/1061086026 wait complete. 2023-12-17T12:09:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-17T12:09:49.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-17T12:08:48.917098+0000', 'modified': '2023-12-17T12:08:55.420903+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24538, 'mds_2': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.141:6835/4280622893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.141:6834', 'nonce': 4280622893}, {'type': 'v1', 'addr': '172.21.15.141:6835', 'nonce': 4280622893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/2634479037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 2634479037}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 2634479037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24509': {'gid': 24509, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6837/1847164381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 1847164381}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 1847164381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.141:6837/3627913634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.141:6836', 'nonce': 3627913634}, {'type': 'v1', 'addr': '172.21.15.141:6837', 'nonce': 3627913634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.192:6839/962693048', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 962693048}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 962693048}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.141:6839/3282544216', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.141:6838', 'nonce': 3282544216}, {'type': 'v1', 'addr': '172.21.15.141:6839', 'nonce': 3282544216}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-17T12:09:49.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-12-17T12:09:49.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-12-17T12:09:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 -- 172.21.15.27:0/2929504886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214104930 msgr2=0x7f9214106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 --2- 172.21.15.27:0/2929504886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214104930 0x7f9214106d20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9200009990 tx=0x7f9200009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 -- 172.21.15.27:0/2929504886 shutdown_connections 2023-12-17T12:09:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 --2- 172.21.15.27:0/2929504886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214102000 0x7f92141043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 --2- 172.21.15.27:0/2929504886 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9214107260 0x7f92141096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 --2- 172.21.15.27:0/2929504886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214104930 0x7f9214106d20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 -- 172.21.15.27:0/2929504886 >> 172.21.15.27:0/2929504886 conn(0x7f92140fb9f0 msgr2=0x7f92140fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 -- 172.21.15.27:0/2929504886 shutdown_connections 2023-12-17T12:09:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.444+0000 7f921b1ff700 1 -- 172.21.15.27:0/2929504886 wait complete. 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.445+0000 7f921b1ff700 1 Processor -- start 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.445+0000 7f921b1ff700 1 -- start start 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921b1ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9214102000 0x7f9214199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921b1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214104930 0x7f9214199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921b1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214107260 0x7f92141a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921b1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92141a1170 con 0x7f9214107260 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921b1ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92141a12b0 con 0x7f9214102000 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921b1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92141a1440 con 0x7f9214104930 2023-12-17T12:09:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921979c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214107260 0x7f92141a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921979c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214107260 0x7f92141a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43590/0 (socket says 172.21.15.27:43590) 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921979c700 1 -- 172.21.15.27:0/1603619326 learned_addr learned my addr 172.21.15.27:0/1603619326 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f9218f9b700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9214102000 0x7f9214199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f9213fff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214104930 0x7f9214199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921979c700 1 -- 172.21.15.27:0/1603619326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214104930 msgr2=0x7f9214199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921979c700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214104930 0x7f9214199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921979c700 1 -- 172.21.15.27:0/1603619326 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9214102000 msgr2=0x7f9214199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921979c700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9214102000 0x7f9214199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.446+0000 7f921979c700 1 -- 172.21.15.27:0/1603619326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000095f0 con 0x7f9214107260 2023-12-17T12:09:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.447+0000 7f921979c700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214107260 0x7f92141a0ba0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f920c007ee0 tx=0x7f920c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.447+0000 7f9211ffb700 1 -- 172.21.15.27:0/1603619326 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c017070 con 0x7f9214107260 2023-12-17T12:09:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.447+0000 7f9211ffb700 1 -- 172.21.15.27:0/1603619326 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f920c00f040 con 0x7f9214107260 2023-12-17T12:09:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.447+0000 7f921b1ff700 1 -- 172.21.15.27:0/1603619326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92141a1720 con 0x7f9214107260 2023-12-17T12:09:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.447+0000 7f9211ffb700 1 -- 172.21.15.27:0/1603619326 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c005710 con 0x7f9214107260 2023-12-17T12:09:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.447+0000 7f921b1ff700 1 -- 172.21.15.27:0/1603619326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92141a1c30 con 0x7f9214107260 2023-12-17T12:09:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.450+0000 7f9211ffb700 1 -- 172.21.15.27:0/1603619326 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f920c005870 con 0x7f9214107260 2023-12-17T12:09:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.451+0000 7f922eadeb80 1 Processor -- start 2023-12-17T12:09:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f922eadeb80 1 -- start start 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f9211ffb700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f92040a1590 0x7f92040a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd643f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 0x555cbd644d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f922eadeb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd5bb420 con 0x555cbd643b00 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f922eadeb80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd59dc00 con 0x555cbd645350 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f922eadeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd59b520 con 0x555cbd644460 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f9211ffb700 1 -- 172.21.15.27:0/1603619326 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f920c0cc720 con 0x7f9214107260 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f9218f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd643f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f9213fff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f92040a1590 0x7f92040a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f9218f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd643f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43602/0 (socket says 172.21.15.27:43602) 2023-12-17T12:09:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.452+0000 7f9218f9b700 1 -- 172.21.15.27:0/3575351818 learned_addr learned my addr 172.21.15.27:0/3575351818 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f9213fff700 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 0x555cbd644d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f9218f9b700 1 -- 172.21.15.27:0/3575351818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 msgr2=0x555cbd644d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f921979c700 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f9218f9b700 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 0x555cbd644d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f9218f9b700 1 -- 172.21.15.27:0/3575351818 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 msgr2=0x555cbd6457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f9218f9b700 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f9218f9b700 1 -- 172.21.15.27:0/3575351818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cbd5f8b50 con 0x555cbd643b00 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f921979c700 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f9213fff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f92040a1590 0x7f92040a3a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9200003780 tx=0x7f920000ba10 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f9218f9b700 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd643f20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f920800bd70 tx=0x7f920800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7af8657eab8185 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f91feffd700 1 -- 172.21.15.27:0/3575351818 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920800fe50 con 0x555cbd643b00 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f91feffd700 1 -- 172.21.15.27:0/3575351818 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f920801a8f0 con 0x555cbd643b00 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.453+0000 7f91feffd700 1 -- 172.21.15.27:0/3575351818 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208011760 con 0x555cbd643b00 2023-12-17T12:09:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 -- 172.21.15.27:0/3575351818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 msgr2=0x555cbd643f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd643f20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f920800bd70 tx=0x7f920800f640 comp rx=0 tx=0).stop 2023-12-17T12:09:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 -- 172.21.15.27:0/3575351818 shutdown_connections 2023-12-17T12:09:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd643f20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3575351818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 0x555cbd644d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 -- 172.21.15.27:0/3575351818 >> 172.21.15.27:0/3575351818 conn(0x555cbd63e670 msgr2=0x555cbd63ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 -- 172.21.15.27:0/3575351818 shutdown_connections 2023-12-17T12:09:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.454+0000 7f922eadeb80 1 -- 172.21.15.27:0/3575351818 wait complete. 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.455+0000 7f922eadeb80 1 -- start start 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.455+0000 7f922eadeb80 1 Processor -- start 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd64d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 0x555cbd64deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd654e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f922eadeb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd596e10 con 0x555cbd643b00 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f922eadeb80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd536000 con 0x555cbd645350 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f922eadeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd535ca0 con 0x555cbd644460 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f922eadeb80 10 client.0 _ll_register_callbacks cb 0x555cbd53e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f9218f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd64d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f9218f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd64d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43614/0 (socket says 172.21.15.27:43614) 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f9218f9b700 1 -- 172.21.15.27:0/2959375302 learned_addr learned my addr 172.21.15.27:0/2959375302 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f921979c700 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd654e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f9218f9b700 1 -- 172.21.15.27:0/2959375302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 msgr2=0x555cbd64deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f9218f9b700 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 0x555cbd64deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.456+0000 7f9218f9b700 1 -- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 msgr2=0x555cbd654e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f9218f9b700 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd654e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f9218f9b700 1 -- 172.21.15.27:0/2959375302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920800b9a0 con 0x555cbd643b00 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f921979c700 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd654e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f9218f9b700 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd64d970 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9208000c00 tx=0x7f920801a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9aeb42972b51255 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f91f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f91f3fff700 1 -- 172.21.15.27:0/2959375302 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208005040 con 0x555cbd643b00 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f91f3fff700 1 -- 172.21.15.27:0/2959375302 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f92080051a0 con 0x555cbd643b00 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f922eadeb80 1 -- 172.21.15.27:0/2959375302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cbd636470 con 0x555cbd643b00 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f91f3fff700 1 -- 172.21.15.27:0/2959375302 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9208005360 con 0x555cbd643b00 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f922eadeb80 10 client.15108 fetch_fsmap learned FSMap version 23 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.457+0000 7f922eadeb80 1 -- 172.21.15.27:0/2959375302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555cbd3bfb10 con 0x555cbd643b00 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f91f3fff700 1 -- 172.21.15.27:0/2959375302 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555cbd636470 con 0x555cbd643b00 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f91f3fff700 1 -- 172.21.15.27:0/2959375302 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9208023030 con 0x555cbd643b00 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f922eadeb80 10 client.15108 fetch_fsmap finished waiting for FSMap version 23 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f922eadeb80 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f922eadeb80 20 client.15108 populate_metadata read hostname 'smithi027' 2023-12-17T12:09:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f922eadeb80 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] conn(0x555cbd659450 0x555cbd657f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f922eadeb80 4 client.15108 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f922eadeb80 1 -- 172.21.15.27:0/2959375302 --> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cbd58fca0 con 0x555cbd659450 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.458+0000 7f9213fff700 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] conn(0x555cbd659450 0x555cbd657f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.459+0000 7f91f3fff700 10 client.15108 ms_handle_connect on v2:172.21.15.141:6836/3627913634 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.459+0000 7f9213fff700 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] conn(0x555cbd659450 0x555cbd657f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fe9e48de9922407 server_cookie=d17c0447040cf8e4 in_seq=0 out_seq=0 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f91f3fff700 1 -- 172.21.15.27:0/2959375302 <== mds.2 v2:172.21.15.141:6836/3627913634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555cbd58fca0 con 0x555cbd659450 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f91f3fff700 10 client.15108 handle_command_reply: tid=0 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 client.15108 shutdown 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 -- 172.21.15.27:0/2959375302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 msgr2=0x555cbd64d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd64d970 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9208000c00 tx=0x7f920801a510 comp rx=0 tx=0).stop 2023-12-17T12:09:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 -- 172.21.15.27:0/2959375302 shutdown_connections 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] conn(0x555cbd659450 0x555cbd657f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd654e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd643b00 0x555cbd64d970 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 --2- 172.21.15.27:0/2959375302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd644460 0x555cbd64deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f922eadeb80 1 -- 172.21.15.27:0/2959375302 >> 172.21.15.27:0/2959375302 conn(0x555cbd63e670 msgr2=0x555cbd63db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.460+0000 7f91f3fff700 0 client.15108 ms_handle_reset on v2:172.21.15.141:6836/3627913634 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.461+0000 7f922eadeb80 1 -- 172.21.15.27:0/2959375302 shutdown_connections 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.461+0000 7f922eadeb80 1 -- 172.21.15.27:0/2959375302 wait complete. 2023-12-17T12:09:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.461+0000 7f922eadeb80 20 client.15108 trim_cache size 0 max 16384 2023-12-17T12:09:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.479+0000 7f922eadeb80 1 Processor -- start 2023-12-17T12:09:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.479+0000 7f922eadeb80 1 -- start start 2023-12-17T12:09:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 0x555cbd645730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd654e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd63a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f922eadeb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd415650 con 0x555cbd644460 2023-12-17T12:09:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f922eadeb80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd5980a0 con 0x555cbd645350 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f922eadeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd592f00 con 0x555cbd643b00 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f9213fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd654e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f9213fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd654e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43616/0 (socket says 172.21.15.27:43616) 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f9213fff700 1 -- 172.21.15.27:0/64643182 learned_addr learned my addr 172.21.15.27:0/64643182 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f9218f9b700 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 0x555cbd645730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f921979c700 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd63a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f9213fff700 1 -- 172.21.15.27:0/64643182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 msgr2=0x555cbd645730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f9213fff700 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 0x555cbd645730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f9213fff700 1 -- 172.21.15.27:0/64643182 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 msgr2=0x555cbd63a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.480+0000 7f9213fff700 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd63a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.481+0000 7f9213fff700 1 -- 172.21.15.27:0/64643182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cbd558290 con 0x555cbd644460 2023-12-17T12:09:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.481+0000 7f9213fff700 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd654e10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9200011040 tx=0x7f920001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbd7d61dbf1bf36 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.481+0000 7f91ff7fe700 1 -- 172.21.15.27:0/64643182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920000b7d0 con 0x555cbd644460 2023-12-17T12:09:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.481+0000 7f91ff7fe700 1 -- 172.21.15.27:0/64643182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9200005700 con 0x555cbd644460 2023-12-17T12:09:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.481+0000 7f91ff7fe700 1 -- 172.21.15.27:0/64643182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200003b10 con 0x555cbd644460 2023-12-17T12:09:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 -- 172.21.15.27:0/64643182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 msgr2=0x555cbd654e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd654e10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9200011040 tx=0x7f920001a050 comp rx=0 tx=0).stop 2023-12-17T12:09:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 -- 172.21.15.27:0/64643182 shutdown_connections 2023-12-17T12:09:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 0x555cbd645730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd63a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 --2- 172.21.15.27:0/64643182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd654e10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 -- 172.21.15.27:0/64643182 >> 172.21.15.27:0/64643182 conn(0x555cbd63e670 msgr2=0x555cbd64b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 -- 172.21.15.27:0/64643182 shutdown_connections 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 -- 172.21.15.27:0/64643182 wait complete. 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.482+0000 7f922eadeb80 1 -- start start 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f922eadeb80 1 Processor -- start 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 0x555cbd652e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd6ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f922eadeb80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f922eadeb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd6ae160 con 0x555cbd644460 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f922eadeb80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd6ae2d0 con 0x555cbd645350 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f922eadeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cbd6ae490 con 0x555cbd643b00 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f922eadeb80 10 client.0 _ll_register_callbacks cb 0x555cbd53e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-17T12:09:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.483+0000 7f9213fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd6ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9213fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd6ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43624/0 (socket says 172.21.15.27:43624) 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9213fff700 1 -- 172.21.15.27:0/3062607161 learned_addr learned my addr 172.21.15.27:0/3062607161 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9218f9b700 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 0x555cbd652e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f921979c700 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9213fff700 1 -- 172.21.15.27:0/3062607161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 msgr2=0x555cbd652e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9213fff700 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 0x555cbd652e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9213fff700 1 -- 172.21.15.27:0/3062607161 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 msgr2=0x555cbd6ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9213fff700 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9213fff700 1 -- 172.21.15.27:0/3062607161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9200020070 con 0x555cbd644460 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f9213fff700 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd6ab4f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9200000c00 tx=0x7f9200009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84178793392fbc87 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f91f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-17T12:09:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f91f3fff700 1 -- 172.21.15.27:0/3062607161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920000b7d0 con 0x555cbd644460 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f91f3fff700 1 -- 172.21.15.27:0/3062607161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9200004500 con 0x555cbd644460 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.484+0000 7f922eadeb80 1 -- 172.21.15.27:0/3062607161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cbd58f9c0 con 0x555cbd644460 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f91f3fff700 1 -- 172.21.15.27:0/3062607161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92000046c0 con 0x555cbd644460 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f922eadeb80 10 client.15114 fetch_fsmap learned FSMap version 23 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f922eadeb80 1 -- 172.21.15.27:0/3062607161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555cbd417c80 con 0x555cbd644460 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f91f3fff700 1 -- 172.21.15.27:0/3062607161 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555cbd58f9c0 con 0x555cbd644460 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f91f3fff700 1 -- 172.21.15.27:0/3062607161 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9200003e20 con 0x555cbd644460 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f922eadeb80 10 client.15114 fetch_fsmap finished waiting for FSMap version 23 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f922eadeb80 10 client.15114 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f922eadeb80 20 client.15114 populate_metadata read hostname 'smithi027' 2023-12-17T12:09:49.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] conn(0x555cbd6b3270 0x555cbd6af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f922eadeb80 4 client.15114 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.485+0000 7f922eadeb80 1 -- 172.21.15.27:0/3062607161 --> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555cbd3e0ca0 con 0x555cbd6b3270 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.486+0000 7f9218f9b700 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] conn(0x555cbd6b3270 0x555cbd6af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.486+0000 7f9218f9b700 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] conn(0x555cbd6b3270 0x555cbd6af760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d28224d17290978 server_cookie=15a051d54a38400d in_seq=0 out_seq=0 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.486+0000 7f91f3fff700 10 client.15114 ms_handle_connect on v2:172.21.15.141:6836/3627913634 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.486+0000 7f91f3fff700 1 -- 172.21.15.27:0/3062607161 <== mds.2 v2:172.21.15.141:6836/3627913634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555cbd3e0ca0 con 0x555cbd6b3270 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.486+0000 7f91f3fff700 10 client.15114 handle_command_reply: tid=0 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.486+0000 7f922eadeb80 1 client.15114 shutdown 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 -- 172.21.15.27:0/3062607161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 msgr2=0x555cbd6ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd6ab4f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9200000c00 tx=0x7f9200009380 comp rx=0 tx=0).stop 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 -- 172.21.15.27:0/3062607161 shutdown_connections 2023-12-17T12:09:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] conn(0x555cbd6b3270 0x555cbd6af760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555cbd643b00 0x555cbd652e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x555cbd645350 0x555cbd6ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f91f3fff700 0 client.15114 ms_handle_reset on v2:172.21.15.141:6836/3627913634 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 --2- 172.21.15.27:0/3062607161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555cbd644460 0x555cbd6ab4f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 -- 172.21.15.27:0/3062607161 >> 172.21.15.27:0/3062607161 conn(0x555cbd63e670 msgr2=0x555cbd63ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 -- 172.21.15.27:0/3062607161 shutdown_connections 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 1 -- 172.21.15.27:0/3062607161 wait complete. 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.487+0000 7f922eadeb80 20 client.15114 trim_cache size 0 max 16384 2023-12-17T12:09:49.494 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "62030760-9cd3-11ee-95a5-87774f69a715", 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24521, 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 55.071599697000003, 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 23, 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 84, 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 76.403463035000001 2023-12-17T12:09:49.495 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-17T12:09:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 -- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f92040a1590 msgr2=0x7f92040a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f92040a1590 0x7f92040a3a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9200003780 tx=0x7f920000ba10 comp rx=0 tx=0).stop 2023-12-17T12:09:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 -- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214107260 msgr2=0x7f92141a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214107260 0x7f92141a0ba0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f920c007ee0 tx=0x7f920c00d3b0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 -- 172.21.15.27:0/1603619326 shutdown_connections 2023-12-17T12:09:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f92040a1590 0x7f92040a3a50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214107260 0x7f92141a0ba0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f9214102000 0x7f9214199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 --2- 172.21.15.27:0/1603619326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214104930 0x7f9214199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 -- 172.21.15.27:0/1603619326 >> 172.21.15.27:0/1603619326 conn(0x7f92140fb9f0 msgr2=0x7f9214100030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 -- 172.21.15.27:0/1603619326 shutdown_connections 2023-12-17T12:09:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.488+0000 7f921b1ff700 1 -- 172.21.15.27:0/1603619326 wait complete. 2023-12-17T12:09:49.508 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-17T12:09:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.716+0000 7fcd1efff700 1 -- 172.21.15.27:0/1372720773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18074030 msgr2=0x7fcd180744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.716+0000 7fcd1efff700 1 --2- 172.21.15.27:0/1372720773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18074030 0x7fcd180744b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009990 tx=0x7fcd04009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.716+0000 7fcd1efff700 1 -- 172.21.15.27:0/1372720773 shutdown_connections 2023-12-17T12:09:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.716+0000 7fcd1efff700 1 --2- 172.21.15.27:0/1372720773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd18075610 0x7fcd18073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.716+0000 7fcd1efff700 1 --2- 172.21.15.27:0/1372720773 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcd18109460 0x7fcd1810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.716+0000 7fcd1efff700 1 --2- 172.21.15.27:0/1372720773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18074030 0x7fcd180744b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.716+0000 7fcd1efff700 1 -- 172.21.15.27:0/1372720773 >> 172.21.15.27:0/1372720773 conn(0x7fcd180fcb10 msgr2=0x7fcd180fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.717+0000 7fcd1efff700 1 -- 172.21.15.27:0/1372720773 shutdown_connections 2023-12-17T12:09:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.717+0000 7fcd1efff700 1 -- 172.21.15.27:0/1372720773 wait complete. 2023-12-17T12:09:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1efff700 1 Processor -- start 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1efff700 1 -- start start 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1efff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd18074030 0x7fcd18199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1efff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcd18075610 0x7fcd18199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1efff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18109460 0x7fcd181a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1efff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181a11b0 con 0x7fcd18109460 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1efff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181a12f0 con 0x7fcd18075610 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1efff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181a1480 con 0x7fcd18074030 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1d59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18109460 0x7fcd181a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1d59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18109460 0x7fcd181a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43634/0 (socket says 172.21.15.27:43634) 2023-12-17T12:09:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1d59c700 1 -- 172.21.15.27:0/356551381 learned_addr learned my addr 172.21.15.27:0/356551381 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.718+0000 7fcd1cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd18074030 0x7fcd18199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd17fff700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcd18075610 0x7fcd18199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd1d59c700 1 -- 172.21.15.27:0/356551381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd18074030 msgr2=0x7fcd18199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd1d59c700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd18074030 0x7fcd18199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd1d59c700 1 -- 172.21.15.27:0/356551381 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcd18075610 msgr2=0x7fcd18199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd1d59c700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcd18075610 0x7fcd18199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd1d59c700 1 -- 172.21.15.27:0/356551381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd040095f0 con 0x7fcd18109460 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd1d59c700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18109460 0x7fcd181a0be0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ef30 tx=0x7fcd1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd15ffb700 1 -- 172.21.15.27:0/356551381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10018070 con 0x7fcd18109460 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd15ffb700 1 -- 172.21.15.27:0/356551381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd1000f040 con 0x7fcd18109460 2023-12-17T12:09:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd1efff700 1 -- 172.21.15.27:0/356551381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd181a1700 con 0x7fcd18109460 2023-12-17T12:09:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd15ffb700 1 -- 172.21.15.27:0/356551381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000cc70 con 0x7fcd18109460 2023-12-17T12:09:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.719+0000 7fcd1efff700 1 -- 172.21.15.27:0/356551381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd181a1bc0 con 0x7fcd18109460 2023-12-17T12:09:49.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.724+0000 7fcd15ffb700 1 -- 172.21.15.27:0/356551381 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fcd10007500 con 0x7fcd18109460 2023-12-17T12:09:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.725+0000 7fcd3295eb80 1 Processor -- start 2023-12-17T12:09:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.725+0000 7fcd3295eb80 1 -- start start 2023-12-17T12:09:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.725+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b64301ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.725+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643020d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.725+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643021350 0x55b6430217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.725+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642f97420 con 0x55b643021350 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.725+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642f79c00 con 0x55b64301fb00 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.725+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642f77520 con 0x55b643020460 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643021350 0x55b6430217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643021350 0x55b6430217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43646/0 (socket says 172.21.15.27:43646) 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 -- 172.21.15.27:0/3416801049 learned_addr learned my addr 172.21.15.27:0/3416801049 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643020d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b64301ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd15ffb700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcd080a1510 0x7fcd080a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 -- 172.21.15.27:0/3416801049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 msgr2=0x55b643020d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643020d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 -- 172.21.15.27:0/3416801049 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 msgr2=0x55b64301ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b64301ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd17fff700 1 -- 172.21.15.27:0/3416801049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b642fd4b50 con 0x55b643021350 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.726+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643020d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.727+0000 7fcd15ffb700 1 -- 172.21.15.27:0/356551381 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcd100cca50 con 0x7fcd18109460 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.727+0000 7fcd1d59c700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcd080a1510 0x7fcd080a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.727+0000 7fcd17fff700 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643021350 0x55b6430217d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009990 tx=0x7fcd0401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642246cffc269715 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.727+0000 7fcd02ffd700 1 -- 172.21.15.27:0/3416801049 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04003960 con 0x55b643021350 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.727+0000 7fcd02ffd700 1 -- 172.21.15.27:0/3416801049 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd0401e070 con 0x55b643021350 2023-12-17T12:09:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.727+0000 7fcd02ffd700 1 -- 172.21.15.27:0/3416801049 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400f460 con 0x55b643021350 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd1d59c700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcd080a1510 0x7fcd080a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd10004b80 tx=0x7fcd10004ad0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3416801049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643021350 msgr2=0x55b6430217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643021350 0x55b6430217d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009990 tx=0x7fcd0401a040 comp rx=0 tx=0).stop 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3416801049 shutdown_connections 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643021350 0x55b6430217d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b64301ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3416801049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643020d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3416801049 >> 172.21.15.27:0/3416801049 conn(0x55b64301a670 msgr2=0x55b64301aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3416801049 shutdown_connections 2023-12-17T12:09:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3416801049 wait complete. 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.728+0000 7fcd3295eb80 1 -- start start 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd3295eb80 1 Processor -- start 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b643029910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b643029e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b643030e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642f72e10 con 0x55b643020460 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642f12000 con 0x55b64301fb00 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642f11ca0 con 0x55b643021350 2023-12-17T12:09:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd3295eb80 10 client.0 _ll_register_callbacks cb 0x55b642f1a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd1cd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b643029e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd1cd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b643029e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43654/0 (socket says 172.21.15.27:43654) 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.729+0000 7fcd1cd9b700 1 -- 172.21.15.27:0/3870393091 learned_addr learned my addr 172.21.15.27:0/3870393091 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd17fff700 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b643030e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd17fff700 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b643029910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd1cd9b700 1 -- 172.21.15.27:0/3870393091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 msgr2=0x55b643030e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b643030e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd1cd9b700 1 -- 172.21.15.27:0/3870393091 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 msgr2=0x55b643029910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b643029910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd1cd9b700 1 -- 172.21.15.27:0/3870393091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c00b8e0 con 0x55b643020460 2023-12-17T12:09:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b643029e50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0042c0 tx=0x7fcd0c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdf2023c3f51c1f server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fccf7fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fccf7fff700 1 -- 172.21.15.27:0/3870393091 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c016070 con 0x55b643020460 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fccf7fff700 1 -- 172.21.15.27:0/3870393091 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcd0c003dc0 con 0x55b643020460 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.730+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3870393091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b643012470 con 0x55b643020460 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fccf7fff700 1 -- 172.21.15.27:0/3870393091 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd0c004830 con 0x55b643020460 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fcd3295eb80 10 client.15126 fetch_fsmap learned FSMap version 23 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3870393091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fcd1810c510 con 0x55b643020460 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fccf7fff700 1 -- 172.21.15.27:0/3870393091 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b643012470 con 0x55b643020460 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fccf7fff700 1 -- 172.21.15.27:0/3870393091 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcd0c024070 con 0x55b643020460 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fcd3295eb80 10 client.15126 fetch_fsmap finished waiting for FSMap version 23 2023-12-17T12:09:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fcd3295eb80 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fcd3295eb80 20 client.15126 populate_metadata read hostname 'smithi027' 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] conn(0x55b6430355a0 0x55b643034090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fcd3295eb80 4 client.15126 mds_command: new command op to 24538 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.731+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3870393091 --> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b642f6bca0 con 0x55b6430355a0 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.732+0000 7fcd17fff700 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] conn(0x55b6430355a0 0x55b643034090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.732+0000 7fcd17fff700 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] conn(0x55b6430355a0 0x55b643034090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4c65bbfe24cde2f server_cookie=baf2981dadfa1fa0 in_seq=0 out_seq=0 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.732+0000 7fccf7fff700 10 client.15126 ms_handle_connect on v2:172.21.15.192:6838/962693048 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.733+0000 7fccf7fff700 1 -- 172.21.15.27:0/3870393091 <== mds.1 v2:172.21.15.192:6838/962693048 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b642f6bca0 con 0x55b6430355a0 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.733+0000 7fccf7fff700 10 client.15126 handle_command_reply: tid=0 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.733+0000 7fcd3295eb80 1 client.15126 shutdown 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3870393091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 msgr2=0x55b643029e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b643029e50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0042c0 tx=0x7fcd0c004680 comp rx=0 tx=0).stop 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3870393091 shutdown_connections 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] conn(0x55b6430355a0 0x55b643034090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b643030e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b643029910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/3870393091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b643029e50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3870393091 >> 172.21.15.27:0/3870393091 conn(0x55b64301a670 msgr2=0x55b643019b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fccf7fff700 0 client.15126 ms_handle_reset on v2:172.21.15.192:6838/962693048 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3870393091 shutdown_connections 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 1 -- 172.21.15.27:0/3870393091 wait complete. 2023-12-17T12:09:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.734+0000 7fcd3295eb80 20 client.15126 trim_cache size 0 max 16384 2023-12-17T12:09:49.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd3295eb80 1 Processor -- start 2023-12-17T12:09:49.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd3295eb80 1 -- start start 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64301fb00 0x55b643028e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643030e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b643021350 0x55b643021730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642df1650 con 0x55b64301fb00 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642eddce0 con 0x55b643021350 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b642f7d6a0 con 0x55b643020460 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd17fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64301fb00 0x55b643028e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd17fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64301fb00 0x55b643028e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43656/0 (socket says 172.21.15.27:43656) 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd17fff700 1 -- 172.21.15.27:0/552436524 learned_addr learned my addr 172.21.15.27:0/552436524 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.758+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643030e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd17fff700 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b643021350 0x55b643021730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd17fff700 1 -- 172.21.15.27:0/552436524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 msgr2=0x55b643030e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd17fff700 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643030e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd17fff700 1 -- 172.21.15.27:0/552436524 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b643021350 msgr2=0x55b643021730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd17fff700 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b643021350 0x55b643021730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd17fff700 1 -- 172.21.15.27:0/552436524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b642d9bb10 con 0x55b64301fb00 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd17fff700 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64301fb00 0x55b643028e20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd040098e0 tx=0x7fcd04005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cdf433a781b809 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd037fe700 1 -- 172.21.15.27:0/552436524 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0402c040 con 0x55b64301fb00 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd037fe700 1 -- 172.21.15.27:0/552436524 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd0400bea0 con 0x55b64301fb00 2023-12-17T12:09:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.759+0000 7fcd037fe700 1 -- 172.21.15.27:0/552436524 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04038800 con 0x55b64301fb00 2023-12-17T12:09:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 -- 172.21.15.27:0/552436524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64301fb00 msgr2=0x55b643028e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64301fb00 0x55b643028e20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd040098e0 tx=0x7fcd04005510 comp rx=0 tx=0).stop 2023-12-17T12:09:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 -- 172.21.15.27:0/552436524 shutdown_connections 2023-12-17T12:09:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b643021350 0x55b643021730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64301fb00 0x55b643028e20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/552436524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643020460 0x55b643030e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 -- 172.21.15.27:0/552436524 >> 172.21.15.27:0/552436524 conn(0x55b64301a670 msgr2=0x55b64301aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 -- 172.21.15.27:0/552436524 shutdown_connections 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.760+0000 7fcd3295eb80 1 -- 172.21.15.27:0/552436524 wait complete. 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 1 -- start start 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 1 Processor -- start 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b643034090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b6430345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b64302fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64302a060 con 0x55b643020460 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64302a1d0 con 0x55b64301fb00 2023-12-17T12:09:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b643030330 con 0x55b643021350 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.761+0000 7fcd3295eb80 10 client.0 _ll_register_callbacks cb 0x55b642f1a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b64302fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b64302fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:59328/0 (socket says 172.21.15.27:59328) 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 -- 172.21.15.27:0/950279222 learned_addr learned my addr 172.21.15.27:0/950279222 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b6430345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b643034090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 -- 172.21.15.27:0/950279222 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 msgr2=0x55b643034090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b643034090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 -- 172.21.15.27:0/950279222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 msgr2=0x55b6430345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b6430345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 -- 172.21.15.27:0/950279222 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b642fd4b50 con 0x55b643021350 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b6430345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fcd17fff700 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b64302fdf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcd040058b0 tx=0x7fcd04009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5b762773eb0a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.762+0000 7fccf7fff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.763+0000 7fccf7fff700 1 -- 172.21.15.27:0/950279222 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0402c040 con 0x55b643021350 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.763+0000 7fccf7fff700 1 -- 172.21.15.27:0/950279222 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcd04000e40 con 0x55b643021350 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.763+0000 7fcd3295eb80 1 -- 172.21.15.27:0/950279222 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd0401e070 con 0x55b643021350 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.763+0000 7fccf7fff700 1 -- 172.21.15.27:0/950279222 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd04003ac0 con 0x55b643021350 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.763+0000 7fcd3295eb80 10 client.24946 fetch_fsmap learned FSMap version 23 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.763+0000 7fcd3295eb80 1 -- 172.21.15.27:0/950279222 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b642f34290 con 0x55b643021350 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.763+0000 7fccf7fff700 1 -- 172.21.15.27:0/950279222 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd0401e070 con 0x55b643021350 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.763+0000 7fccf7fff700 1 -- 172.21.15.27:0/950279222 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcd040099c0 con 0x55b643021350 2023-12-17T12:09:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.764+0000 7fcd3295eb80 10 client.24946 fetch_fsmap finished waiting for FSMap version 23 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.764+0000 7fcd3295eb80 10 client.24946 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.764+0000 7fcd3295eb80 20 client.24946 populate_metadata read hostname 'smithi027' 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.764+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] conn(0x55b64308f3a0 0x55b64308b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.764+0000 7fcd3295eb80 4 client.24946 mds_command: new command op to 24538 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.764+0000 7fcd3295eb80 1 -- 172.21.15.27:0/950279222 --> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b642f6b9c0 con 0x55b64308f3a0 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.764+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] conn(0x55b64308f3a0 0x55b64308b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd1cd9b700 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] conn(0x55b64308f3a0 0x55b64308b860 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f86cc9a7faa71385 server_cookie=836a0230bbc5c5dc in_seq=0 out_seq=0 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fccf7fff700 10 client.24946 ms_handle_connect on v2:172.21.15.192:6838/962693048 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fccf7fff700 1 -- 172.21.15.27:0/950279222 <== mds.1 v2:172.21.15.192:6838/962693048 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b642f6b9c0 con 0x55b64308f3a0 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fccf7fff700 10 client.24946 handle_command_reply: tid=0 2023-12-17T12:09:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 client.24946 shutdown 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 -- 172.21.15.27:0/950279222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 msgr2=0x55b64302fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b64302fdf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcd040058b0 tx=0x7fcd04009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 -- 172.21.15.27:0/950279222 shutdown_connections 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] conn(0x55b64308f3a0 0x55b64308b860 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b643021350 0x55b64302fdf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x55b64301fb00 0x55b643034090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fccf7fff700 0 client.24946 ms_handle_reset on v2:172.21.15.192:6838/962693048 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 --2- 172.21.15.27:0/950279222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b643020460 0x55b6430345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 -- 172.21.15.27:0/950279222 >> 172.21.15.27:0/950279222 conn(0x55b64301a670 msgr2=0x55b64303df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 -- 172.21.15.27:0/950279222 shutdown_connections 2023-12-17T12:09:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.765+0000 7fcd3295eb80 1 -- 172.21.15.27:0/950279222 wait complete. 2023-12-17T12:09:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd3295eb80 20 client.24946 trim_cache size 0 max 16384 2023-12-17T12:09:49.773 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-17T12:09:49.773 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "62030760-9cd3-11ee-95a5-87774f69a715", 2023-12-17T12:09:49.773 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2023-12-17T12:09:49.773 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24538, 2023-12-17T12:09:49.773 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-17T12:09:49.773 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-17T12:09:49.774 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-17T12:09:49.774 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 56.358615743999998, 2023-12-17T12:09:49.774 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 23, 2023-12-17T12:09:49.774 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 84, 2023-12-17T12:09:49.774 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2023-12-17T12:09:49.774 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 74.029182263999999 2023-12-17T12:09:49.774 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 -- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcd080a1510 msgr2=0x7fcd080a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcd080a1510 0x7fcd080a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd10004b80 tx=0x7fcd10004ad0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 -- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18109460 msgr2=0x7fcd181a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18109460 0x7fcd181a0be0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ef30 tx=0x7fcd1000c3a0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 -- 172.21.15.27:0/356551381 shutdown_connections 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcd080a1510 0x7fcd080a39d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd18074030 0x7fcd18199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd18109460 0x7fcd181a0be0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 --2- 172.21.15.27:0/356551381 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcd18075610 0x7fcd18199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:49.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 -- 172.21.15.27:0/356551381 >> 172.21.15.27:0/356551381 conn(0x7fcd180fcb10 msgr2=0x7fcd18105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:49.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 -- 172.21.15.27:0/356551381 shutdown_connections 2023-12-17T12:09:49.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:49.766+0000 7fcd1efff700 1 -- 172.21.15.27:0/356551381 wait complete. 2023-12-17T12:09:49.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:49 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4019953570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:49.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:49 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1061086026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:49.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-17T12:09:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.010+0000 7f2d299ff700 1 -- 172.21.15.27:0/3982163437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 msgr2=0x7f2d24073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.010+0000 7f2d299ff700 1 --2- 172.21.15.27:0/3982163437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 0x7f2d24073a60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009990 tx=0x7f2d14009ca0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.011+0000 7f2d299ff700 1 -- 172.21.15.27:0/3982163437 shutdown_connections 2023-12-17T12:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.011+0000 7f2d299ff700 1 --2- 172.21.15.27:0/3982163437 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2d24109460 0x7f2d2410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.011+0000 7f2d299ff700 1 --2- 172.21.15.27:0/3982163437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 0x7f2d24073a60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.011+0000 7f2d299ff700 1 --2- 172.21.15.27:0/3982163437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d24074030 0x7f2d240744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.011+0000 7f2d299ff700 1 -- 172.21.15.27:0/3982163437 >> 172.21.15.27:0/3982163437 conn(0x7f2d240fcb10 msgr2=0x7f2d240fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.011+0000 7f2d299ff700 1 -- 172.21.15.27:0/3982163437 shutdown_connections 2023-12-17T12:09:50.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.011+0000 7f2d299ff700 1 -- 172.21.15.27:0/3982163437 wait complete. 2023-12-17T12:09:50.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:49 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4019953570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:50.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:49 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1061086026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d299ff700 1 Processor -- start 2023-12-17T12:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d299ff700 1 -- start start 2023-12-17T12:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d299ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2d24074030 0x7f2d241996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d299ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 0x7f2d24199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d299ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d24109460 0x7f2d241a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d299ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241a11d0 con 0x7f2d24075610 2023-12-17T12:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d299ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241a1310 con 0x7f2d24074030 2023-12-17T12:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d299ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241a14a0 con 0x7f2d24109460 2023-12-17T12:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d227fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 0x7f2d24199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d22ffd700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2d24074030 0x7f2d241996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d237fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d24109460 0x7f2d241a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d227fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 0x7f2d24199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43672/0 (socket says 172.21.15.27:43672) 2023-12-17T12:09:50.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.012+0000 7f2d227fc700 1 -- 172.21.15.27:0/2268845530 learned_addr learned my addr 172.21.15.27:0/2268845530 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d237fe700 1 -- 172.21.15.27:0/2268845530 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2d24074030 msgr2=0x7f2d241996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d237fe700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2d24074030 0x7f2d241996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d237fe700 1 -- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 msgr2=0x7f2d24199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d237fe700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 0x7f2d24199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d237fe700 1 -- 172.21.15.27:0/2268845530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140095f0 con 0x7f2d24109460 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d227fc700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 0x7f2d24199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d237fe700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d24109460 0x7f2d241a0c00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00efc0 tx=0x7f2d1c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d289fd700 1 -- 172.21.15.27:0/2268845530 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c018070 con 0x7f2d24109460 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d299ff700 1 -- 172.21.15.27:0/2268845530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d241a1720 con 0x7f2d24109460 2023-12-17T12:09:50.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d289fd700 1 -- 172.21.15.27:0/2268845530 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d1c00f040 con 0x7f2d24109460 2023-12-17T12:09:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.013+0000 7f2d289fd700 1 -- 172.21.15.27:0/2268845530 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00cba0 con 0x7f2d24109460 2023-12-17T12:09:50.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.014+0000 7f2d299ff700 1 -- 172.21.15.27:0/2268845530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d241a1c30 con 0x7f2d24109460 2023-12-17T12:09:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.017+0000 7f2d289fd700 1 -- 172.21.15.27:0/2268845530 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2d1c003a00 con 0x7f2d24109460 2023-12-17T12:09:50.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d3d2e5b80 1 Processor -- start 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d3d2e5b80 1 -- start start 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d289fd700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2d0c0a1510 0x7f2d0c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb45f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 0x563debb46d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x563debabd420 con 0x563debb47350 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deba9fc00 con 0x563debb46460 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d289fd700 1 -- 172.21.15.27:0/2268845530 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d1c0cc9e0 con 0x7f2d24109460 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.018+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deba9d520 con 0x563debb45b00 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d22ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb45f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d227fc700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2d0c0a1510 0x7f2d0c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d22ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb45f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:59358/0 (socket says 172.21.15.27:59358) 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d22ffd700 1 -- 172.21.15.27:0/2424971248 learned_addr learned my addr 172.21.15.27:0/2424971248 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d227fc700 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 0x563debb46d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d22ffd700 1 -- 172.21.15.27:0/2424971248 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 msgr2=0x563debb46d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 0x563debb46d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d22ffd700 1 -- 172.21.15.27:0/2424971248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 msgr2=0x563debb477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d22ffd700 1 -- 172.21.15.27:0/2424971248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563debafab50 con 0x563debb45b00 2023-12-17T12:09:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.019+0000 7f2d237fe700 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.020+0000 7f2d227fc700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2d0c0a1510 0x7f2d0c0a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009880 tx=0x7f2d180047c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.020+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb45f20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d14003f60 tx=0x7f2d14005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca016ef0a03282b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.020+0000 7f2d09ffb700 1 -- 172.21.15.27:0/2424971248 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1401d070 con 0x563debb45b00 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.020+0000 7f2d09ffb700 1 -- 172.21.15.27:0/2424971248 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d1401e040 con 0x563debb45b00 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.020+0000 7f2d09ffb700 1 -- 172.21.15.27:0/2424971248 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400b710 con 0x563debb45b00 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/2424971248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 msgr2=0x563debb45f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb45f20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d14003f60 tx=0x7f2d14005ec0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/2424971248 shutdown_connections 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb45f20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/2424971248 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 0x563debb46d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/2424971248 >> 172.21.15.27:0/2424971248 conn(0x563debb40670 msgr2=0x563debb40a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/2424971248 shutdown_connections 2023-12-17T12:09:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/2424971248 wait complete. 2023-12-17T12:09:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.021+0000 7f2d3d2e5b80 1 -- start start 2023-12-17T12:09:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d3d2e5b80 1 Processor -- start 2023-12-17T12:09:50.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb4f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb46460 0x563debb4fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb56de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deba98e10 con 0x563debb47350 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deba38000 con 0x563debb45b00 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deba37ca0 con 0x563debb46460 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d3d2e5b80 10 client.0 _ll_register_callbacks cb 0x563deba403a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d22ffd700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d22ffd700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb4f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:51592/0 (socket says 172.21.15.27:51592) 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb56de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d22ffd700 1 -- 172.21.15.27:0/1770926676 learned_addr learned my addr 172.21.15.27:0/1770926676 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.022+0000 7f2d227fc700 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb46460 0x563debb4fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d22ffd700 1 -- 172.21.15.27:0/1770926676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb46460 msgr2=0x563debb4fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb46460 0x563debb4fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d22ffd700 1 -- 172.21.15.27:0/1770926676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 msgr2=0x563debb56de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb56de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d22ffd700 1 -- 172.21.15.27:0/1770926676 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d241a1c30 con 0x563debb45b00 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d237fe700 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb56de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb4f860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009ce0 tx=0x7f2d14003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe10a5f0f5be2168 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.141:3300/0 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.023+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1770926676 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1401d040 con 0x563debb45b00 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1770926676 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563debb38470 con 0x563debb45b00 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1770926676 <== mon.2 v2:172.21.15.141:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d1400b910 con 0x563debb45b00 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1770926676 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d1400bad0 con 0x563debb45b00 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d3d2e5b80 10 client.24944 fetch_fsmap learned FSMap version 23 2023-12-17T12:09:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1770926676 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f2d2410c510 con 0x563debb45b00 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1770926676 <== mon.2 v2:172.21.15.141:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563debb38470 con 0x563debb45b00 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1770926676 <== mon.2 v2:172.21.15.141:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2d14022400 con 0x563debb45b00 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d3d2e5b80 10 client.24944 fetch_fsmap finished waiting for FSMap version 23 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d3d2e5b80 10 client.24944 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d3d2e5b80 20 client.24944 populate_metadata read hostname 'smithi027' 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.024+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] conn(0x563debb5b040 0x563debb59db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.025+0000 7f2d3d2e5b80 4 client.24944 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.025+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1770926676 --> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563deba91ca0 con 0x563debb5b040 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.025+0000 7f2d227fc700 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] conn(0x563debb5b040 0x563debb59db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.026+0000 7f2d02ffd700 10 client.24944 ms_handle_connect on v2:172.21.15.141:6838/3282544216 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.026+0000 7f2d227fc700 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] conn(0x563debb5b040 0x563debb59db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c06df921e65fcd8 server_cookie=24aa2c412dd6b6b2 in_seq=0 out_seq=0 2023-12-17T12:09:50.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1770926676 <== mds.0 v2:172.21.15.141:6838/3282544216 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563deba91ca0 con 0x563debb5b040 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d02ffd700 10 client.24944 handle_command_reply: tid=0 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 client.24944 shutdown 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1770926676 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 msgr2=0x563debb4f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb4f860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009ce0 tx=0x7f2d14003890 comp rx=0 tx=0).stop 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1770926676 shutdown_connections 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb56de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb4f860 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] conn(0x563debb5b040 0x563debb59db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1770926676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb46460 0x563debb4fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d02ffd700 0 client.24944 ms_handle_reset on v2:172.21.15.141:6838/3282544216 2023-12-17T12:09:50.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.027+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1770926676 >> 172.21.15.27:0/1770926676 conn(0x563debb40670 msgr2=0x563debb3fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.028+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1770926676 shutdown_connections 2023-12-17T12:09:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.028+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1770926676 wait complete. 2023-12-17T12:09:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.028+0000 7f2d3d2e5b80 20 client.24944 trim_cache size 0 max 16384 2023-12-17T12:09:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.046+0000 7f2d3d2e5b80 1 Processor -- start 2023-12-17T12:09:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d3d2e5b80 1 -- start start 2023-12-17T12:09:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb57490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb47350 0x563debb4d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deb917650 con 0x563debb46460 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x563debb58af0 con 0x563debb45b00 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563debb46a70 con 0x563debb47350 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d227fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d22ffd700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb57490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d227fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43694/0 (socket says 172.21.15.27:43694) 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d237fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb47350 0x563debb4d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.047+0000 7f2d227fc700 1 -- 172.21.15.27:0/100300365 learned_addr learned my addr 172.21.15.27:0/100300365 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d22ffd700 1 -- 172.21.15.27:0/100300365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb47350 msgr2=0x563debb4d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/100300365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb47350 0x563debb4d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d22ffd700 1 -- 172.21.15.27:0/100300365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb46460 msgr2=0x563debb56d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:50.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/100300365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d22ffd700 1 -- 172.21.15.27:0/100300365 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563deb8c1b10 con 0x563debb45b00 2023-12-17T12:09:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d227fc700 1 --2- 172.21.15.27:0/100300365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/100300365 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb57490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009990 tx=0x7f2d1400b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db54a22d35779b1f server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d0a7fc700 1 -- 172.21.15.27:0/100300365 <== mon.2 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14003a20 con 0x563debb45b00 2023-12-17T12:09:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d0a7fc700 1 -- 172.21.15.27:0/100300365 <== mon.2 v2:172.21.15.141:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d14022b00 con 0x563debb45b00 2023-12-17T12:09:50.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.048+0000 7f2d0a7fc700 1 -- 172.21.15.27:0/100300365 <== mon.2 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400f750 con 0x563debb45b00 2023-12-17T12:09:50.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/100300365 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 msgr2=0x563debb57490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/100300365 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb57490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009990 tx=0x7f2d1400b610 comp rx=0 tx=0).stop 2023-12-17T12:09:50.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/100300365 shutdown_connections 2023-12-17T12:09:50.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/100300365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb47350 0x563debb4d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/100300365 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb45b00 0x563debb57490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/100300365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/100300365 >> 172.21.15.27:0/100300365 conn(0x563debb40670 msgr2=0x563debb4ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/100300365 shutdown_connections 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.049+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/100300365 wait complete. 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.050+0000 7f2d3d2e5b80 1 -- start start 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.050+0000 7f2d3d2e5b80 1 Processor -- start 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb4c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d3d2e5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb4c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x563debb500f0 con 0x563debb47350 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x563debb50260 con 0x563debb46460 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d3d2e5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563debb4ccd0 con 0x563debb45b00 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d3d2e5b80 10 client.0 _ll_register_callbacks cb 0x563deba403a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d22ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb4c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d22ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb4c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:59388/0 (socket says 172.21.15.27:59388) 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d22ffd700 1 -- 172.21.15.27:0/1791168488 learned_addr learned my addr 172.21.15.27:0/1791168488 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d227fc700 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb4c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d22ffd700 1 -- 172.21.15.27:0/1791168488 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 msgr2=0x563debb56d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d22ffd700 1 -- 172.21.15.27:0/1791168488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 msgr2=0x563debb4c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb4c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d22ffd700 1 -- 172.21.15.27:0/1791168488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d18018070 con 0x563debb45b00 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.051+0000 7f2d237fe700 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb4c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.052+0000 7f2d22ffd700 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb4c250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2d140098e0 tx=0x7f2d140226b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f72fac88d91cb6d server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.052+0000 7f2d02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.052+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1791168488 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1401d070 con 0x563debb45b00 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.052+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1791168488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d1401e040 con 0x563debb45b00 2023-12-17T12:09:50.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.052+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1791168488 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d14000e40 con 0x563debb45b00 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.052+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1791168488 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d14003a20 con 0x563debb45b00 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.052+0000 7f2d3d2e5b80 10 client.24964 fetch_fsmap learned FSMap version 23 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.052+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1791168488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f2d2410c510 con 0x563debb45b00 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1791168488 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d14052080 con 0x563debb45b00 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1791168488 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2d1403b450 con 0x563debb45b00 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d3d2e5b80 10 client.24964 fetch_fsmap finished waiting for FSMap version 23 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d3d2e5b80 10 client.24964 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d3d2e5b80 20 client.24964 populate_metadata read hostname 'smithi027' 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] conn(0x563debbb5a70 0x563debbb1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:50.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d3d2e5b80 4 client.24964 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1791168488 --> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563deba919c0 con 0x563debbb5a70 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.053+0000 7f2d227fc700 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] conn(0x563debbb5a70 0x563debbb1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.054+0000 7f2d227fc700 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] conn(0x563debbb5a70 0x563debbb1f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f451bc957324d076 server_cookie=58384cb9cf8abcdb in_seq=0 out_seq=0 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.054+0000 7f2d02ffd700 10 client.24964 ms_handle_connect on v2:172.21.15.141:6838/3282544216 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.054+0000 7f2d02ffd700 1 -- 172.21.15.27:0/1791168488 <== mds.0 v2:172.21.15.141:6838/3282544216 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563deba919c0 con 0x563debbb5a70 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.054+0000 7f2d02ffd700 10 client.24964 handle_command_reply: tid=0 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.054+0000 7f2d3d2e5b80 1 client.24964 shutdown 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1791168488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 msgr2=0x563debb4c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb4c250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2d140098e0 tx=0x7f2d140226b0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1791168488 shutdown_connections 2023-12-17T12:09:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] conn(0x563debbb5a70 0x563debbb1f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563debb45b00 0x563debb4c250 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563debb47350 0x563debb4c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 --2- 172.21.15.27:0/1791168488 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x563debb46460 0x563debb56d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1791168488 >> 172.21.15.27:0/1791168488 conn(0x563debb40670 msgr2=0x563debb4d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d02ffd700 0 client.24964 ms_handle_reset on v2:172.21.15.141:6838/3282544216 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1791168488 shutdown_connections 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 1 -- 172.21.15.27:0/1791168488 wait complete. 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d3d2e5b80 20 client.24964 trim_cache size 0 max 16384 2023-12-17T12:09:50.062 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "62030760-9cd3-11ee-95a5-87774f69a715", 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24566, 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 60.176116294000003, 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 23, 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 84, 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 66.421760524000007 2023-12-17T12:09:50.063 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-17T12:09:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d299ff700 1 -- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2d0c0a1510 msgr2=0x7f2d0c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.055+0000 7f2d299ff700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2d0c0a1510 0x7f2d0c0a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009880 tx=0x7f2d180047c0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 -- 172.21.15.27:0/2268845530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d24109460 msgr2=0x7f2d241a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d24109460 0x7f2d241a0c00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00efc0 tx=0x7f2d1c00c3a0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 -- 172.21.15.27:0/2268845530 shutdown_connections 2023-12-17T12:09:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f2d0c0a1510 0x7f2d0c0a39d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d24109460 0x7f2d241a0c00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f2d24074030 0x7f2d241996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 --2- 172.21.15.27:0/2268845530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d24075610 0x7f2d24199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 -- 172.21.15.27:0/2268845530 >> 172.21.15.27:0/2268845530 conn(0x7f2d240fcb10 msgr2=0x7f2d24105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 -- 172.21.15.27:0/2268845530 shutdown_connections 2023-12-17T12:09:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:50.056+0000 7f2d299ff700 1 -- 172.21.15.27:0/2268845530 wait complete. 2023-12-17T12:09:50.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:49 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4019953570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:09:50.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:49 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1061086026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:09:50.076 INFO:tasks.cephadm:Setup complete, yielding 2023-12-17T12:09:50.076 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-17T12:09:50.086 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2023-12-17T12:09:50.087 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- bash -c 'ceph orch status' 2023-12-17T12:09:50.702 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:50.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:50 smithi027 ceph-mon[36871]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:50.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:50 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:09:51.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:50 smithi141 ceph-mon[41225]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:51.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:50 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:09:51.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:50 smithi192 ceph-mon[41899]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:51.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:50 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-17T12:09:52.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 -- 172.21.15.27:0/2993908795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c41031c0 msgr2=0x7ff9c4109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:52.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/2993908795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c41031c0 0x7ff9c4109a70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00b0d0 tx=0x7ff9bc00b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 -- 172.21.15.27:0/2993908795 shutdown_connections 2023-12-17T12:09:52.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/2993908795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c41031c0 0x7ff9c4109a70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/2993908795 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff9c4101620 0x7ff9c4101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/2993908795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c4102820 0x7ff9c4102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 -- 172.21.15.27:0/2993908795 >> 172.21.15.27:0/2993908795 conn(0x7ff9c40fcbe0 msgr2=0x7ff9c40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:52.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 -- 172.21.15.27:0/2993908795 shutdown_connections 2023-12-17T12:09:52.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.081+0000 7ff9c8da9700 1 -- 172.21.15.27:0/2993908795 wait complete. 2023-12-17T12:09:52.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.083+0000 7ff9c8da9700 1 Processor -- start 2023-12-17T12:09:52.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.083+0000 7ff9c8da9700 1 -- start start 2023-12-17T12:09:52.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.083+0000 7ff9c8da9700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff9c4101620 0x7ff9c41996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:52.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.083+0000 7ff9c8da9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c4102820 0x7ff9c4199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:52.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c8da9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c41031c0 0x7ff9c41a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:52.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c8da9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a1230 con 0x7ff9c4102820 2023-12-17T12:09:52.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c8da9700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a1370 con 0x7ff9c4101620 2023-12-17T12:09:52.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c8da9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a1500 con 0x7ff9c41031c0 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c259c700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff9c4101620 0x7ff9c41996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c1d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c4102820 0x7ff9c4199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c259c700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff9c4101620 0x7ff9c41996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.27:51624/0 (socket says 172.21.15.27:51624) 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c259c700 1 -- 172.21.15.27:0/814500672 learned_addr learned my addr 172.21.15.27:0/814500672 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c2d9d700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c41031c0 0x7ff9c41a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c1d9b700 1 -- 172.21.15.27:0/814500672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c41031c0 msgr2=0x7ff9c41a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c1d9b700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c41031c0 0x7ff9c41a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c1d9b700 1 -- 172.21.15.27:0/814500672 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff9c4101620 msgr2=0x7ff9c41996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c1d9b700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff9c4101620 0x7ff9c41996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c1d9b700 1 -- 172.21.15.27:0/814500672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc009d00 con 0x7ff9c4102820 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.084+0000 7ff9c259c700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff9c4101620 0x7ff9c41996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.085+0000 7ff9c1d9b700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c4102820 0x7ff9c4199c30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800ea20 tx=0x7ff9b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.085+0000 7ff9b37fe700 1 -- 172.21.15.27:0/814500672 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800cbd0 con 0x7ff9c4102820 2023-12-17T12:09:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.085+0000 7ff9b37fe700 1 -- 172.21.15.27:0/814500672 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9b800ee60 con 0x7ff9c4102820 2023-12-17T12:09:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.085+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c41a17e0 con 0x7ff9c4102820 2023-12-17T12:09:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.085+0000 7ff9b37fe700 1 -- 172.21.15.27:0/814500672 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8018420 con 0x7ff9c4102820 2023-12-17T12:09:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.085+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c41a1d30 con 0x7ff9c4102820 2023-12-17T12:09:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.086+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c404f290 con 0x7ff9c4102820 2023-12-17T12:09:52.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.090+0000 7ff9b37fe700 1 -- 172.21.15.27:0/814500672 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff9b8010430 con 0x7ff9c4102820 2023-12-17T12:09:52.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.091+0000 7ff9b37fe700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff9ac0a1590 0x7ff9ac0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:52.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.092+0000 7ff9b37fe700 1 -- 172.21.15.27:0/814500672 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff9b8014070 con 0x7ff9c4102820 2023-12-17T12:09:52.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.092+0000 7ff9c259c700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff9ac0a1590 0x7ff9ac0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:52.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.093+0000 7ff9c259c700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff9ac0a1590 0x7ff9ac0a3a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005950 tx=0x7ff9b4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:52.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.096+0000 7ff9b37fe700 1 -- 172.21.15.27:0/814500672 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff9b8096410 con 0x7ff9c4102820 2023-12-17T12:09:52.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.366+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9c4074f20 con 0x7ff9ac0a1590 2023-12-17T12:09:52.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.368+0000 7ff9b37fe700 1 -- 172.21.15.27:0/814500672 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff9c4074f20 con 0x7ff9ac0a1590 2023-12-17T12:09:52.369 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2023-12-17T12:09:52.369 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2023-12-17T12:09:52.369 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.370+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff9ac0a1590 msgr2=0x7ff9ac0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.370+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff9ac0a1590 0x7ff9ac0a3a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005950 tx=0x7ff9b4009260 comp rx=0 tx=0).stop 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.370+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c4102820 msgr2=0x7ff9c4199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.370+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c4102820 0x7ff9c4199c30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800ea20 tx=0x7ff9b800ed30 comp rx=0 tx=0).stop 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.371+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 shutdown_connections 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.371+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff9ac0a1590 0x7ff9ac0a3a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.371+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c41031c0 0x7ff9c41a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.371+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff9c4101620 0x7ff9c41996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.371+0000 7ff9c8da9700 1 --2- 172.21.15.27:0/814500672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c4102820 0x7ff9c4199c30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.371+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 >> 172.21.15.27:0/814500672 conn(0x7ff9c40fcbe0 msgr2=0x7ff9c4073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.371+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 shutdown_connections 2023-12-17T12:09:52.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:52.371+0000 7ff9c8da9700 1 -- 172.21.15.27:0/814500672 wait complete. 2023-12-17T12:09:52.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:52 smithi027 ceph-mon[36871]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-17T12:09:52.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:52 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:52 smithi192 ceph-mon[41899]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-17T12:09:52.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:52 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:52 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:52 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:52 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:52 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:52 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:52.832 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- bash -c 'ceph orch ps' 2023-12-17T12:09:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:52 smithi141 ceph-mon[41225]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-17T12:09:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:52 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:52 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:52 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:52 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:52 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:53.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:52 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:53.291 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:53.827 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:53 smithi027 ceph-mon[36871]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:53.827 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:53 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:09:53.827 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:53 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:09:53.827 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:53 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:54.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:53 smithi141 ceph-mon[41225]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:54.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:53 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:09:54.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:53 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:09:54.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:53 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:54.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:53 smithi192 ceph-mon[41899]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:54.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:53 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-17T12:09:54.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:53 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-17T12:09:54.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:53 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' 2023-12-17T12:09:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.575+0000 7fbaf2c35700 1 -- 172.21.15.27:0/2061798751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec102820 msgr2=0x7fbaec102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.575+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/2061798751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec102820 0x7fbaec102c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbad80099c0 tx=0x7fbad8009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.575+0000 7fbaf2c35700 1 -- 172.21.15.27:0/2061798751 shutdown_connections 2023-12-17T12:09:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.575+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/2061798751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec1031c0 0x7fbaec109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.575+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/2061798751 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbaec101620 0x7fbaec101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.575+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/2061798751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec102820 0x7fbaec102c80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.575+0000 7fbaf2c35700 1 -- 172.21.15.27:0/2061798751 >> 172.21.15.27:0/2061798751 conn(0x7fbaec0fcbe0 msgr2=0x7fbaec0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.576+0000 7fbaf2c35700 1 -- 172.21.15.27:0/2061798751 shutdown_connections 2023-12-17T12:09:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.576+0000 7fbaf2c35700 1 -- 172.21.15.27:0/2061798751 wait complete. 2023-12-17T12:09:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.577+0000 7fbaf2c35700 1 Processor -- start 2023-12-17T12:09:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.577+0000 7fbaf2c35700 1 -- start start 2023-12-17T12:09:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.577+0000 7fbaf2c35700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec101620 0x7fbaec1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.577+0000 7fbaf2c35700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec102820 0x7fbaec197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.577+0000 7fbaf2c35700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbaec1031c0 0x7fbaec19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.577+0000 7fbaf2c35700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec19f000 con 0x7fbaec101620 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.577+0000 7fbaf2c35700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec19f140 con 0x7fbaec1031c0 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.577+0000 7fbaf2c35700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec19f2d0 con 0x7fbaec102820 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec101620 0x7fbaec1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec101620 0x7fbaec1974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43738/0 (socket says 172.21.15.27:43738) 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 -- 172.21.15.27:0/4235262049 learned_addr learned my addr 172.21.15.27:0/4235262049 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaebfff700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec102820 0x7fbaec197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf11d2700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbaec1031c0 0x7fbaec19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 -- 172.21.15.27:0/4235262049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec102820 msgr2=0x7fbaec197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec102820 0x7fbaec197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 -- 172.21.15.27:0/4235262049 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbaec1031c0 msgr2=0x7fbaec19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbaec1031c0 0x7fbaec19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 -- 172.21.15.27:0/4235262049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80095f0 con 0x7fbaec101620 2023-12-17T12:09:54.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.578+0000 7fbaf09d1700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec101620 0x7fbaec1974c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbae000e9a0 tx=0x7fbae000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:54.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.579+0000 7fbae9ffb700 1 -- 172.21.15.27:0/4235262049 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0009790 con 0x7fbaec101620 2023-12-17T12:09:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.579+0000 7fbae9ffb700 1 -- 172.21.15.27:0/4235262049 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbae000c810 con 0x7fbaec101620 2023-12-17T12:09:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.579+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaec19f5b0 con 0x7fbaec101620 2023-12-17T12:09:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.579+0000 7fbae9ffb700 1 -- 172.21.15.27:0/4235262049 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0003bd0 con 0x7fbaec101620 2023-12-17T12:09:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.579+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec19fb00 con 0x7fbaec101620 2023-12-17T12:09:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.580+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaec108ba0 con 0x7fbaec101620 2023-12-17T12:09:54.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.582+0000 7fbae9ffb700 1 -- 172.21.15.27:0/4235262049 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fbae000c980 con 0x7fbaec101620 2023-12-17T12:09:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.584+0000 7fbae9ffb700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbadc0a1590 0x7fbadc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.584+0000 7fbae9ffb700 1 -- 172.21.15.27:0/4235262049 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbae0014070 con 0x7fbaec101620 2023-12-17T12:09:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.584+0000 7fbaebfff700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbadc0a1590 0x7fbadc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.585+0000 7fbaebfff700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbadc0a1590 0x7fbadc0a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbad8000c00 tx=0x7fbad8003680 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.588+0000 7fbae9ffb700 1 -- 172.21.15.27:0/4235262049 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbae00964e0 con 0x7fbaec101620 2023-12-17T12:09:54.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:54 smithi027 ceph-mon[36871]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:54.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.807+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaec061a10 con 0x7fbadc0a1590 2023-12-17T12:09:54.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.821+0000 7fbae9ffb700 1 -- 172.21.15.27:0/4235262049 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7fbaec061a10 con 0x7fbadc0a1590 2023-12-17T12:09:54.822 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.a smithi027 running (73s) 2s ago 73s 13.8M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 24d0e0dfcacb 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.b smithi141 running (71s) 3s ago 71s 29.0M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 9145a4c3781e 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.c smithi192 running (84s) 3s ago 84s 18.5M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 baa644251777 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.d smithi027 running (92s) 2s ago 92s 13.8M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 7db036f35d19 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.e smithi141 running (89s) 3s ago 89s 17.3M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 48eacc27a9f1 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.f smithi192 running (87s) 3s ago 86s 17.3M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 58540c2b702a 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.g smithi027 running (76s) 2s ago 76s 13.8M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 e841b8463b72 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.h smithi141 running (82s) 3s ago 81s 18.2M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 5232468c94e3 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.i smithi192 running (79s) 3s ago 79s 18.0M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 bb38209c02e3 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.j smithi027 running (94s) 2s ago 94s 13.8M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 282d98a64b33 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.k smithi141 running (97s) 3s ago 97s 13.9M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 a59bf74a49b0 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mds.l smithi192 running (100s) 3s ago 99s 13.9M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 977b7e2de95a 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.x smithi027 *:9283,8765 running (8m) 2s ago 8m 529M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 64628503b641 2023-12-17T12:09:54.823 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.y smithi141 *:8443,8765 running (6m) 3s ago 6m 452M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 80bf96e71c55 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.z smithi192 *:8443,8765 running (6m) 3s ago 6m 452M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 b77d5f30b108 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a smithi027 running (8m) 2s ago 8m 93.6M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 db878f6c8f52 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b smithi141 running (6m) 3s ago 6m 61.1M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 93d686ce02bf 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c smithi192 running (6m) 3s ago 6m 60.5M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 e573025f2aba 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (6m) 2s ago 6m 67.5M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 78fdb0f43b3a 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi141 running (5m) 3s ago 5m 74.7M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 5dc3c3214dbb 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi192 running (5m) 3s ago 5m 71.2M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 b6043e752e8f 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi027 running (4m) 2s ago 4m 61.8M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 e328965ab9d1 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi141 running (4m) 3s ago 4m 60.7M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 9ab02b7ead0a 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi192 running (4m) 3s ago 4m 63.3M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 9de4a44b7501 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 smithi027 running (3m) 2s ago 3m 58.8M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 eef90ce7dc06 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 smithi141 running (3m) 3s ago 3m 62.2M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 6f4dc16fbcaa 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 smithi192 running (3m) 3s ago 3m 59.5M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 2ee21488bc1a 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 smithi027 running (2m) 2s ago 2m 55.9M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 a1b632972fc4 2023-12-17T12:09:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 smithi141 running (2m) 3s ago 2m 60.4M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 361318a498b7 2023-12-17T12:09:54.825 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 smithi192 running (119s) 3s ago 119s 51.8M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 11da08abff1f 2023-12-17T12:09:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.823+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbadc0a1590 msgr2=0x7fbadc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.823+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbadc0a1590 0x7fbadc0a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbad8000c00 tx=0x7fbad8003680 comp rx=0 tx=0).stop 2023-12-17T12:09:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec101620 msgr2=0x7fbaec1974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:54.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec101620 0x7fbaec1974c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbae000e9a0 tx=0x7fbae000ecb0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 shutdown_connections 2023-12-17T12:09:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fbadc0a1590 0x7fbadc0a3a50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbaec1031c0 0x7fbaec19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec101620 0x7fbaec1974c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 --2- 172.21.15.27:0/4235262049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec102820 0x7fbaec197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 >> 172.21.15.27:0/4235262049 conn(0x7fbaec0fcbe0 msgr2=0x7fbaec105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 shutdown_connections 2023-12-17T12:09:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:54.824+0000 7fbaf2c35700 1 -- 172.21.15.27:0/4235262049 wait complete. 2023-12-17T12:09:55.013 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:54 smithi141 ceph-mon[41225]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:55.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:54 smithi192 ceph-mon[41899]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:09:55.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- bash -c 'ceph orch ls' 2023-12-17T12:09:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:56.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:56 smithi027 ceph-mon[36871]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:56.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:56 smithi027 ceph-mon[36871]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:09:56.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 -- 172.21.15.27:0/2883298018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 msgr2=0x7f19e4101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 --2- 172.21.15.27:0/2883298018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 0x7f19e4101a40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f19d40099c0 tx=0x7f19d4009cd0 comp rx=0 tx=0).stop 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 -- 172.21.15.27:0/2883298018 shutdown_connections 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 --2- 172.21.15.27:0/2883298018 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f19e41031c0 0x7f19e4109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 --2- 172.21.15.27:0/2883298018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 0x7f19e4101a40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 --2- 172.21.15.27:0/2883298018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e4102820 0x7f19e4102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 -- 172.21.15.27:0/2883298018 >> 172.21.15.27:0/2883298018 conn(0x7f19e40fcbe0 msgr2=0x7f19e40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 -- 172.21.15.27:0/2883298018 shutdown_connections 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.924+0000 7f19e98af700 1 -- 172.21.15.27:0/2883298018 wait complete. 2023-12-17T12:09:56.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.926+0000 7f19e98af700 1 Processor -- start 2023-12-17T12:09:56.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.926+0000 7f19e98af700 1 -- start start 2023-12-17T12:09:56.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.926+0000 7f19e98af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 0x7f19e4197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:56.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.926+0000 7f19e98af700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e4102820 0x7f19e4197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:56.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.926+0000 7f19e98af700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f19e41031c0 0x7f19e419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:56.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e98af700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419ef80 con 0x7f19e4101620 2023-12-17T12:09:56.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e98af700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419f0c0 con 0x7f19e41031c0 2023-12-17T12:09:56.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e98af700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419f250 con 0x7f19e4102820 2023-12-17T12:09:56.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 0x7f19e4197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 0x7f19e4197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51948/0 (socket says 172.21.15.27:51948) 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e2ffd700 1 -- 172.21.15.27:0/2784676855 learned_addr learned my addr 172.21.15.27:0/2784676855 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e37fe700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f19e41031c0 0x7f19e419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e27fc700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e4102820 0x7f19e4197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e2ffd700 1 -- 172.21.15.27:0/2784676855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e4102820 msgr2=0x7f19e4197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e2ffd700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e4102820 0x7f19e4197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e2ffd700 1 -- 172.21.15.27:0/2784676855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f19e41031c0 msgr2=0x7f19e419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e2ffd700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f19e41031c0 0x7f19e419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.927+0000 7f19e2ffd700 1 -- 172.21.15.27:0/2784676855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40095f0 con 0x7f19e4101620 2023-12-17T12:09:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.928+0000 7f19e2ffd700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 0x7f19e4197440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009990 tx=0x7f19d4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:56.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.928+0000 7f19e88ad700 1 -- 172.21.15.27:0/2784676855 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d401d070 con 0x7f19e4101620 2023-12-17T12:09:56.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.928+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e419f4d0 con 0x7f19e4101620 2023-12-17T12:09:56.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.928+0000 7f19e88ad700 1 -- 172.21.15.27:0/2784676855 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19d400bda0 con 0x7f19e4101620 2023-12-17T12:09:56.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.928+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e419f8f0 con 0x7f19e4101620 2023-12-17T12:09:56.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.928+0000 7f19e88ad700 1 -- 172.21.15.27:0/2784676855 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4022c70 con 0x7f19e4101620 2023-12-17T12:09:56.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.929+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e4108b00 con 0x7f19e4101620 2023-12-17T12:09:56.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.931+0000 7f19e88ad700 1 -- 172.21.15.27:0/2784676855 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f19d4022470 con 0x7f19e4101620 2023-12-17T12:09:56.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.933+0000 7f19e88ad700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19cc0a1510 0x7f19cc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:56.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.933+0000 7f19e88ad700 1 -- 172.21.15.27:0/2784676855 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19d40cdf90 con 0x7f19e4101620 2023-12-17T12:09:56.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.933+0000 7f19e27fc700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19cc0a1510 0x7f19cc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:56.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.934+0000 7f19e27fc700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19cc0a1510 0x7f19cc0a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005950 tx=0x7f19d800b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:56.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:56.937+0000 7f19e88ad700 1 -- 172.21.15.27:0/2784676855 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f19d405d2f0 con 0x7f19e4101620 2023-12-17T12:09:57.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:56 smithi141 ceph-mon[41225]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:57.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:56 smithi141 ceph-mon[41225]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:09:57.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:56 smithi192 ceph-mon[41899]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:57.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:56 smithi192 ceph-mon[41899]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:09:57.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.157+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f19e4061a10 con 0x7f19cc0a1510 2023-12-17T12:09:57.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.163+0000 7f19e88ad700 1 -- 172.21.15.27:0/2784676855 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f19e4061a10 con 0x7f19cc0a1510 2023-12-17T12:09:57.165 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-12-17T12:09:57.165 INFO:teuthology.orchestra.run.smithi027.stdout:mds.all 12/12 5s ago 105s smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi141=b;smithi141=e;smithi141=h;smithi141=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-12-17T12:09:57.165 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 3/3 5s ago 6m smithi027=x;smithi141=y;smithi192=z;count:3 2023-12-17T12:09:57.165 INFO:teuthology.orchestra.run.smithi027.stdout:mon 3/3 5s ago 6m smithi027:172.21.15.27=a;smithi141:172.21.15.141=b;smithi192:172.21.15.192=c;count:3 2023-12-17T12:09:57.165 INFO:teuthology.orchestra.run.smithi027.stdout:osd 12 5s ago - 2023-12-17T12:09:57.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19cc0a1510 msgr2=0x7f19cc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:57.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19cc0a1510 0x7f19cc0a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005950 tx=0x7f19d800b410 comp rx=0 tx=0).stop 2023-12-17T12:09:57.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 msgr2=0x7f19e4197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 0x7f19e4197440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009990 tx=0x7f19d4005b00 comp rx=0 tx=0).stop 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 shutdown_connections 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f19cc0a1510 0x7f19cc0a39d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f19e41031c0 0x7f19e419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19e4101620 0x7f19e4197440 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 --2- 172.21.15.27:0/2784676855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e4102820 0x7f19e4197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 >> 172.21.15.27:0/2784676855 conn(0x7f19e40fcbe0 msgr2=0x7f19e4105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 shutdown_connections 2023-12-17T12:09:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:57.166+0000 7f19e98af700 1 -- 172.21.15.27:0/2784676855 wait complete. 2023-12-17T12:09:57.633 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- bash -c 'ceph orch host ls' 2023-12-17T12:09:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:09:58.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:58 smithi027 ceph-mon[36871]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:09:58.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:09:58 smithi027 ceph-mon[36871]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:59.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:58 smithi141 ceph-mon[41225]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:09:59.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:09:58 smithi141 ceph-mon[41225]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:59.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:58 smithi192 ceph-mon[41899]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:09:59.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:09:58 smithi192 ceph-mon[41899]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:09:59.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.342+0000 7f312b135700 1 -- 172.21.15.27:0/3152619351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31241031c0 msgr2=0x7f3124109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:59.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.342+0000 7f312b135700 1 --2- 172.21.15.27:0/3152619351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31241031c0 0x7f3124109a70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f311c00b0d0 tx=0x7f311c00b3e0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.342+0000 7f312b135700 1 -- 172.21.15.27:0/3152619351 shutdown_connections 2023-12-17T12:09:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.342+0000 7f312b135700 1 --2- 172.21.15.27:0/3152619351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3124101620 0x7f3124101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.342+0000 7f312b135700 1 --2- 172.21.15.27:0/3152619351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31241031c0 0x7f3124109a70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.342+0000 7f312b135700 1 --2- 172.21.15.27:0/3152619351 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3124102820 0x7f3124102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.342+0000 7f312b135700 1 -- 172.21.15.27:0/3152619351 >> 172.21.15.27:0/3152619351 conn(0x7f31240fcbe0 msgr2=0x7f31240ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.343+0000 7f312b135700 1 -- 172.21.15.27:0/3152619351 shutdown_connections 2023-12-17T12:09:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.343+0000 7f312b135700 1 -- 172.21.15.27:0/3152619351 wait complete. 2023-12-17T12:09:59.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.344+0000 7f312b135700 1 Processor -- start 2023-12-17T12:09:59.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.344+0000 7f312b135700 1 -- start start 2023-12-17T12:09:59.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.344+0000 7f312b135700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3124101620 0x7f3124197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:59.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.344+0000 7f312b135700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3124102820 0x7f3124197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:59.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.344+0000 7f312b135700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f31241031c0 0x7f312419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:59.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.344+0000 7f312b135700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312419f090 con 0x7f3124101620 2023-12-17T12:09:59.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.344+0000 7f312b135700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312419f1d0 con 0x7f31241031c0 2023-12-17T12:09:59.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.344+0000 7f312b135700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312419f360 con 0x7f3124102820 2023-12-17T12:09:59.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3124101620 0x7f3124197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:59.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3124101620 0x7f3124197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51966/0 (socket says 172.21.15.27:51966) 2023-12-17T12:09:59.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 -- 172.21.15.27:0/1915935608 learned_addr learned my addr 172.21.15.27:0/1915935608 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:09:59.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3123fff700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3124102820 0x7f3124197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f31296d2700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f31241031c0 0x7f312419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 -- 172.21.15.27:0/1915935608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3124102820 msgr2=0x7f3124197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3124102820 0x7f3124197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 -- 172.21.15.27:0/1915935608 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f31241031c0 msgr2=0x7f312419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f31241031c0 0x7f312419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 -- 172.21.15.27:0/1915935608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c009d00 con 0x7f3124101620 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f31296d2700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f31241031c0 0x7f312419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.345+0000 7f3128ed1700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3124101620 0x7f3124197550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f311800d9b0 tx=0x7f311800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.346+0000 7f3121ffb700 1 -- 172.21.15.27:0/1915935608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311800be80 con 0x7f3124101620 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.346+0000 7f3121ffb700 1 -- 172.21.15.27:0/1915935608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31180044b0 con 0x7f3124101620 2023-12-17T12:09:59.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.346+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312419f640 con 0x7f3124101620 2023-12-17T12:09:59.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.346+0000 7f3121ffb700 1 -- 172.21.15.27:0/1915935608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118003aa0 con 0x7f3124101620 2023-12-17T12:09:59.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.346+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312419fb90 con 0x7f3124101620 2023-12-17T12:09:59.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.347+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3124108ab0 con 0x7f3124101620 2023-12-17T12:09:59.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.349+0000 7f3121ffb700 1 -- 172.21.15.27:0/1915935608 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f3118009940 con 0x7f3124101620 2023-12-17T12:09:59.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.350+0000 7f3121ffb700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f31140a1590 0x7f31140a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:09:59.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.350+0000 7f3121ffb700 1 -- 172.21.15.27:0/1915935608 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f31180ccb80 con 0x7f3124101620 2023-12-17T12:09:59.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.351+0000 7f3123fff700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f31140a1590 0x7f31140a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:09:59.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.351+0000 7f3123fff700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f31140a1590 0x7f31140a3a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3110005950 tx=0x7f31100058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:09:59.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.355+0000 7f3121ffb700 1 -- 172.21.15.27:0/1915935608 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3118096030 con 0x7f3124101620 2023-12-17T12:09:59.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.588+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f312402d380 con 0x7f31140a1590 2023-12-17T12:09:59.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.589+0000 7f3121ffb700 1 -- 172.21.15.27:0/1915935608 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f312402d380 con 0x7f31140a1590 2023-12-17T12:09:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2023-12-17T12:09:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2023-12-17T12:09:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:smithi141 172.21.15.141 2023-12-17T12:09:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:smithi192 172.21.15.192 2023-12-17T12:09:59.591 INFO:teuthology.orchestra.run.smithi027.stdout:3 hosts in cluster 2023-12-17T12:09:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f31140a1590 msgr2=0x7f31140a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f31140a1590 0x7f31140a3a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3110005950 tx=0x7f31100058e0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3124101620 msgr2=0x7f3124197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:09:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3124101620 0x7f3124197550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f311800d9b0 tx=0x7f311800dd70 comp rx=0 tx=0).stop 2023-12-17T12:09:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 shutdown_connections 2023-12-17T12:09:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f31140a1590 0x7f31140a3a50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f31241031c0 0x7f312419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3124101620 0x7f3124197550 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.591+0000 7f312b135700 1 --2- 172.21.15.27:0/1915935608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3124102820 0x7f3124197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:09:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.592+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 >> 172.21.15.27:0/1915935608 conn(0x7f31240fcbe0 msgr2=0x7f3124105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:09:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.592+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 shutdown_connections 2023-12-17T12:09:59.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:09:59.592+0000 7f312b135700 1 -- 172.21.15.27:0/1915935608 wait complete. 2023-12-17T12:10:00.098 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- bash -c 'ceph orch device ls' 2023-12-17T12:10:00.534 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:10:00.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:00 smithi027 ceph-mon[36871]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:00.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:00 smithi027 ceph-mon[36871]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:10:00.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:00 smithi027 ceph-mon[36871]: overall HEALTH_OK 2023-12-17T12:10:01.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:00 smithi141 ceph-mon[41225]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:01.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:00 smithi141 ceph-mon[41225]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:10:01.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:00 smithi141 ceph-mon[41225]: overall HEALTH_OK 2023-12-17T12:10:01.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:00 smithi192 ceph-mon[41899]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:01.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:00 smithi192 ceph-mon[41899]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:10:01.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:00 smithi192 ceph-mon[41899]: overall HEALTH_OK 2023-12-17T12:10:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 -- 172.21.15.27:0/1465625511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 msgr2=0x7fec50101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 --2- 172.21.15.27:0/1465625511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 0x7fec50101a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0099c0 tx=0x7fec4c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:10:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 -- 172.21.15.27:0/1465625511 shutdown_connections 2023-12-17T12:10:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 --2- 172.21.15.27:0/1465625511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec501031c0 0x7fec50109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 --2- 172.21.15.27:0/1465625511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 0x7fec50101a40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:01.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 --2- 172.21.15.27:0/1465625511 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec50102820 0x7fec50102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:01.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 -- 172.21.15.27:0/1465625511 >> 172.21.15.27:0/1465625511 conn(0x7fec500fcbe0 msgr2=0x7fec500ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:01.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 -- 172.21.15.27:0/1465625511 shutdown_connections 2023-12-17T12:10:01.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.782+0000 7fec58272700 1 -- 172.21.15.27:0/1465625511 wait complete. 2023-12-17T12:10:01.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.783+0000 7fec58272700 1 Processor -- start 2023-12-17T12:10:01.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec58272700 1 -- start start 2023-12-17T12:10:01.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec58272700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 0x7fec50199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:01.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec58272700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec50102820 0x7fec50199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:01.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec58272700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec501031c0 0x7fec501a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:01.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec58272700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec501a1250 con 0x7fec50101620 2023-12-17T12:10:01.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec58272700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec501a1390 con 0x7fec50102820 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec58272700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec501a1520 con 0x7fec501031c0 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec5600e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 0x7fec50199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec5600e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 0x7fec50199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51982/0 (socket says 172.21.15.27:51982) 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec5600e700 1 -- 172.21.15.27:0/4177295010 learned_addr learned my addr 172.21.15.27:0/4177295010 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec5580d700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec50102820 0x7fec50199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.784+0000 7fec5680f700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec501031c0 0x7fec501a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec5600e700 1 -- 172.21.15.27:0/4177295010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec501031c0 msgr2=0x7fec501a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec5600e700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec501031c0 0x7fec501a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec5600e700 1 -- 172.21.15.27:0/4177295010 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec50102820 msgr2=0x7fec50199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec5600e700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec50102820 0x7fec50199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:01.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec5600e700 1 -- 172.21.15.27:0/4177295010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c0095f0 con 0x7fec50101620 2023-12-17T12:10:01.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec5600e700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 0x7fec50199760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec4c000c00 tx=0x7fec4c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:01.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec477fe700 1 -- 172.21.15.27:0/4177295010 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c01d070 con 0x7fec50101620 2023-12-17T12:10:01.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec477fe700 1 -- 172.21.15.27:0/4177295010 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec4c003e20 con 0x7fec50101620 2023-12-17T12:10:01.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec501a17a0 con 0x7fec50101620 2023-12-17T12:10:01.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec477fe700 1 -- 172.21.15.27:0/4177295010 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00fc30 con 0x7fec50101620 2023-12-17T12:10:01.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.785+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec501a1c10 con 0x7fec50101620 2023-12-17T12:10:01.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.787+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5004f290 con 0x7fec50101620 2023-12-17T12:10:01.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.788+0000 7fec477fe700 1 -- 172.21.15.27:0/4177295010 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fec4c003990 con 0x7fec50101620 2023-12-17T12:10:01.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.790+0000 7fec477fe700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec3c0a1590 0x7fec3c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:01.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.790+0000 7fec477fe700 1 -- 172.21.15.27:0/4177295010 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fec4c0cdc30 con 0x7fec50101620 2023-12-17T12:10:01.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.790+0000 7fec5580d700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec3c0a1590 0x7fec3c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:01.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.791+0000 7fec5580d700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec3c0a1590 0x7fec3c0a3a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fec40005950 tx=0x7fec4000b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:01.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:01.794+0000 7fec477fe700 1 -- 172.21.15.27:0/4177295010 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fec4c097170 con 0x7fec50101620 2023-12-17T12:10:02.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.010+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 --> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fec50109760 con 0x7fec3c0a1590 2023-12-17T12:10:02.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.012+0000 7fec477fe700 1 -- 172.21.15.27:0/4177295010 <== mgr.14154 v2:172.21.15.27:6800/46170456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fec50109760 con 0x7fec3c0a1590 2023-12-17T12:10:02.013 INFO:teuthology.orchestra.run.smithi027.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-12-17T12:10:02.013 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-17T12:10:02.013 INFO:teuthology.orchestra.run.smithi027.stdout:smithi141 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6242001S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-17T12:10:02.013 INFO:teuthology.orchestra.run.smithi027.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 119s ago Has a FileSystem, LVM detected 2023-12-17T12:10:02.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec3c0a1590 msgr2=0x7fec3c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:02.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec3c0a1590 0x7fec3c0a3a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fec40005950 tx=0x7fec4000b410 comp rx=0 tx=0).stop 2023-12-17T12:10:02.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 msgr2=0x7fec50199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:02.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 0x7fec50199760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec4c000c00 tx=0x7fec4c0045e0 comp rx=0 tx=0).stop 2023-12-17T12:10:02.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 shutdown_connections 2023-12-17T12:10:02.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fec3c0a1590 0x7fec3c0a3a50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:02.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec501031c0 0x7fec501a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:02.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec50101620 0x7fec50199760 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:02.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 --2- 172.21.15.27:0/4177295010 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fec50102820 0x7fec50199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:02.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 >> 172.21.15.27:0/4177295010 conn(0x7fec500fcbe0 msgr2=0x7fec50108040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:02.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 shutdown_connections 2023-12-17T12:10:02.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:02.014+0000 7fec58272700 1 -- 172.21.15.27:0/4177295010 wait complete. 2023-12-17T12:10:02.435 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-17T12:10:02.444 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2023-12-17T12:10:02.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- bash -c 'ceph fs dump' 2023-12-17T12:10:02.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:02 smithi027 ceph-mon[36871]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-17T12:10:02.875 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:10:03.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:02 smithi141 ceph-mon[41225]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-17T12:10:03.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:02 smithi192 ceph-mon[41899]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-17T12:10:03.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:03 smithi027 ceph-mon[36871]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:10:04.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:03 smithi141 ceph-mon[41225]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:10:04.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:03 smithi192 ceph-mon[41899]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-17T12:10:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.139+0000 7ff64050a700 1 -- 172.21.15.27:0/2687359697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff638101620 msgr2=0x7ff638101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.139+0000 7ff64050a700 1 --2- 172.21.15.27:0/2687359697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff638101620 0x7ff638101a40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff6340099c0 tx=0x7ff634009cd0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.140+0000 7ff64050a700 1 -- 172.21.15.27:0/2687359697 shutdown_connections 2023-12-17T12:10:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.140+0000 7ff64050a700 1 --2- 172.21.15.27:0/2687359697 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff6381031c0 0x7ff638109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.140+0000 7ff64050a700 1 --2- 172.21.15.27:0/2687359697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff638101620 0x7ff638101a40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.140+0000 7ff64050a700 1 --2- 172.21.15.27:0/2687359697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff638102820 0x7ff638102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.140+0000 7ff64050a700 1 -- 172.21.15.27:0/2687359697 >> 172.21.15.27:0/2687359697 conn(0x7ff6380fcbe0 msgr2=0x7ff6380ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:04.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.140+0000 7ff64050a700 1 -- 172.21.15.27:0/2687359697 shutdown_connections 2023-12-17T12:10:04.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.140+0000 7ff64050a700 1 -- 172.21.15.27:0/2687359697 wait complete. 2023-12-17T12:10:04.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.141+0000 7ff64050a700 1 Processor -- start 2023-12-17T12:10:04.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.141+0000 7ff64050a700 1 -- start start 2023-12-17T12:10:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff64050a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff638101620 0x7ff6381974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff64050a700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff638102820 0x7ff638197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff64050a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6381031c0 0x7ff63819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff64050a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63819f010 con 0x7ff6381031c0 2023-12-17T12:10:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff64050a700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63819f150 con 0x7ff638102820 2023-12-17T12:10:04.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff64050a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63819f2e0 con 0x7ff638101620 2023-12-17T12:10:04.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff63eaa7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6381031c0 0x7ff63819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:04.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff63eaa7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6381031c0 0x7ff63819ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52012/0 (socket says 172.21.15.27:52012) 2023-12-17T12:10:04.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff63eaa7700 1 -- 172.21.15.27:0/3470219147 learned_addr learned my addr 172.21.15.27:0/3470219147 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:04.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff63daa5700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff638102820 0x7ff638197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:04.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.142+0000 7ff63e2a6700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff638101620 0x7ff6381974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:04.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff63eaa7700 1 -- 172.21.15.27:0/3470219147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff638101620 msgr2=0x7ff6381974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:04.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff63eaa7700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff638101620 0x7ff6381974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff63eaa7700 1 -- 172.21.15.27:0/3470219147 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff638102820 msgr2=0x7ff638197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:04.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff63eaa7700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff638102820 0x7ff638197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff63eaa7700 1 -- 172.21.15.27:0/3470219147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340095f0 con 0x7ff6381031c0 2023-12-17T12:10:04.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff63eaa7700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6381031c0 0x7ff63819ea40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff630007f10 tx=0x7ff63000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:04.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff62f7fe700 1 -- 172.21.15.27:0/3470219147 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630017070 con 0x7ff6381031c0 2023-12-17T12:10:04.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63819f5c0 con 0x7ff6381031c0 2023-12-17T12:10:04.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff62f7fe700 1 -- 172.21.15.27:0/3470219147 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff63000f040 con 0x7ff6381031c0 2023-12-17T12:10:04.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.143+0000 7ff62f7fe700 1 -- 172.21.15.27:0/3470219147 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630005570 con 0x7ff6381031c0 2023-12-17T12:10:04.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.144+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63819fb10 con 0x7ff6381031c0 2023-12-17T12:10:04.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.145+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff638108ba0 con 0x7ff6381031c0 2023-12-17T12:10:04.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.146+0000 7ff62f7fe700 1 -- 172.21.15.27:0/3470219147 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff63000db20 con 0x7ff6381031c0 2023-12-17T12:10:04.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.148+0000 7ff62f7fe700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff6240a1590 0x7ff6240a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:04.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.148+0000 7ff62f7fe700 1 -- 172.21.15.27:0/3470219147 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff6300cc540 con 0x7ff6381031c0 2023-12-17T12:10:04.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.148+0000 7ff63e2a6700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff6240a1590 0x7ff6240a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.149+0000 7ff63e2a6700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff6240a1590 0x7ff6240a3a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff634006010 tx=0x7ff634003740 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.152+0000 7ff62f7fe700 1 -- 172.21.15.27:0/3470219147 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff6300959f0 con 0x7ff6381031c0 2023-12-17T12:10:04.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.410+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff63810cec0 con 0x7ff6381031c0 2023-12-17T12:10:04.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.411+0000 7ff62f7fe700 1 -- 172.21.15.27:0/3470219147 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2947 (secure 0 0 0) 0x7ff630099010 con 0x7ff6381031c0 2023-12-17T12:10:04.412 INFO:teuthology.orchestra.run.smithi027.stdout:e23 2023-12-17T12:10:04.412 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-12-17T12:10:04.412 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-17T12:10:04.412 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2023-12-17T12:10:04.412 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:10:04.412 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2023-12-17T12:10:04.412 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 23 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:created 2023-12-17T12:08:48.917098+0000 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2023-12-17T12:08:55.420903+0000 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 300 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:max_xattr_size 65536 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-17T12:10:04.413 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 3 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1,2 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24566,1=24538,2=24521} 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:bal_rank_mask -1 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b{0:24566} state up:active seq 3 addr [v2:172.21.15.141:6838/3282544216,v1:172.21.15.141:6839/3282544216] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c{0:24509} state up:standby-replay seq 1 addr [v2:172.21.15.192:6836/1847164381,v1:172.21.15.192:6837/1847164381] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.i{1:24538} state up:active seq 6 addr [v2:172.21.15.192:6838/962693048,v1:172.21.15.192:6839/962693048] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f{1:24502} state up:standby-replay seq 1 addr [v2:172.21.15.192:6834/2634479037,v1:172.21.15.192:6835/2634479037] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.414 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.h{2:24521} state up:active seq 7 addr [v2:172.21.15.141:6836/3627913634,v1:172.21.15.141:6837/3627913634] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e{2:24488} state up:standby-replay seq 1 addr [v2:172.21.15.141:6834/4280622893,v1:172.21.15.141:6835/4280622893] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d{-1:14616} state up:standby seq 1 addr [v2:172.21.15.27:6836/2608720361,v1:172.21.15.27:6837/2608720361] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.g{-1:14688} state up:standby seq 1 addr [v2:172.21.15.27:6838/1099412442,v1:172.21.15.27:6839/1099412442] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a{-1:14700} state up:standby seq 1 addr [v2:172.21.15.27:6840/3551652066,v1:172.21.15.27:6841/3551652066] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.l{-1:24448} state up:standby seq 1 addr [v2:172.21.15.192:6832/2710858125,v1:172.21.15.192:6833/2710858125] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.k{-1:24458} state up:standby seq 1 addr [v2:172.21.15.141:6832/2577446255,v1:172.21.15.141:6833/2577446255] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.415 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.j{-1:24472} state up:standby seq 1 addr [v2:172.21.15.27:6834/2531226190,v1:172.21.15.27:6835/2531226190] compat {c=[1],r=[1],i=[fff]}] 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff6240a1590 msgr2=0x7ff6240a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff6240a1590 0x7ff6240a3a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff634006010 tx=0x7ff634003740 comp rx=0 tx=0).stop 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6381031c0 msgr2=0x7ff63819ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6381031c0 0x7ff63819ea40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff630007f10 tx=0x7ff63000d3b0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 shutdown_connections 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7ff6240a1590 0x7ff6240a3a50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff638101620 0x7ff6381974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6381031c0 0x7ff63819ea40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 --2- 172.21.15.27:0/3470219147 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7ff638102820 0x7ff638197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:04.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 >> 172.21.15.27:0/3470219147 conn(0x7ff6380fcbe0 msgr2=0x7ff638105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:04.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 shutdown_connections 2023-12-17T12:10:04.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:04.413+0000 7ff64050a700 1 -- 172.21.15.27:0/3470219147 wait complete. 2023-12-17T12:10:04.417 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-17T12:10:04.786 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:04 smithi027 ceph-mon[36871]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:04.786 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:04 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/3470219147' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-17T12:10:04.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62030760-9cd3-11ee-95a5-87774f69a715 -- bash -c 'ceph osd dump' 2023-12-17T12:10:05.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:04 smithi141 ceph-mon[41225]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:05.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:04 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/3470219147' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-17T12:10:05.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:04 smithi192 ceph-mon[41899]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:05.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:04 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/3470219147' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-17T12:10:05.250 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/mon.a/config 2023-12-17T12:10:06.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 -- 172.21.15.27:0/2369278182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18102820 msgr2=0x7fcc18102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 --2- 172.21.15.27:0/2369278182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18102820 0x7fcc18102c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0099c0 tx=0x7fcc0c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 -- 172.21.15.27:0/2369278182 shutdown_connections 2023-12-17T12:10:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 --2- 172.21.15.27:0/2369278182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc18101620 0x7fcc18101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 --2- 172.21.15.27:0/2369278182 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcc181031c0 0x7fcc18109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 --2- 172.21.15.27:0/2369278182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18102820 0x7fcc18102c80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 -- 172.21.15.27:0/2369278182 >> 172.21.15.27:0/2369278182 conn(0x7fcc180fcbe0 msgr2=0x7fcc180ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 -- 172.21.15.27:0/2369278182 shutdown_connections 2023-12-17T12:10:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.544+0000 7fcc1e411700 1 -- 172.21.15.27:0/2369278182 wait complete. 2023-12-17T12:10:06.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.545+0000 7fcc1e411700 1 Processor -- start 2023-12-17T12:10:06.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.545+0000 7fcc1e411700 1 -- start start 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.545+0000 7fcc1e411700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18101620 0x7fcc181996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.545+0000 7fcc1e411700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcc18102820 0x7fcc18199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc1e411700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc181031c0 0x7fcc181a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc1e411700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181a11e0 con 0x7fcc18101620 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc1e411700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181a1320 con 0x7fcc18102820 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc1e411700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181a14b0 con 0x7fcc181031c0 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc17fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18101620 0x7fcc181996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc177fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcc18102820 0x7fcc18199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc17fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18101620 0x7fcc181996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42952/0 (socket says 172.21.15.27:42952) 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc17fff700 1 -- 172.21.15.27:0/1220977460 learned_addr learned my addr 172.21.15.27:0/1220977460 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc1c9ae700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc181031c0 0x7fcc181a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc17fff700 1 -- 172.21.15.27:0/1220977460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc181031c0 msgr2=0x7fcc181a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc17fff700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc181031c0 0x7fcc181a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc17fff700 1 -- 172.21.15.27:0/1220977460 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcc18102820 msgr2=0x7fcc18199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc17fff700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcc18102820 0x7fcc18199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.546+0000 7fcc17fff700 1 -- 172.21.15.27:0/1220977460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0095f0 con 0x7fcc18101620 2023-12-17T12:10:06.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.547+0000 7fcc17fff700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18101620 0x7fcc181996a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800d9b0 tx=0x7fcc0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:06.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.547+0000 7fcc157fa700 1 -- 172.21.15.27:0/1220977460 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08004020 con 0x7fcc18101620 2023-12-17T12:10:06.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.547+0000 7fcc157fa700 1 -- 172.21.15.27:0/1220977460 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc08009730 con 0x7fcc18101620 2023-12-17T12:10:06.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.547+0000 7fcc157fa700 1 -- 172.21.15.27:0/1220977460 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc080039c0 con 0x7fcc18101620 2023-12-17T12:10:06.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.547+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc181a1790 con 0x7fcc18101620 2023-12-17T12:10:06.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.547+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc181a1c40 con 0x7fcc18101620 2023-12-17T12:10:06.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.548+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1804f290 con 0x7fcc18101620 2023-12-17T12:10:06.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.550+0000 7fcc157fa700 1 -- 172.21.15.27:0/1220977460 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fcc080098a0 con 0x7fcc18101620 2023-12-17T12:10:06.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.552+0000 7fcc157fa700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcc000a1510 0x7fcc000a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:06.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.553+0000 7fcc157fa700 1 -- 172.21.15.27:0/1220977460 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcc080ccbd0 con 0x7fcc18101620 2023-12-17T12:10:06.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.553+0000 7fcc177fe700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcc000a1510 0x7fcc000a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:06.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.553+0000 7fcc177fe700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcc000a1510 0x7fcc000a39d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00b580 tx=0x7fcc0c005e80 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:06.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.557+0000 7fcc157fa700 1 -- 172.21.15.27:0/1220977460 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcc08096080 con 0x7fcc18101620 2023-12-17T12:10:06.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.768+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fcc1810cf60 con 0x7fcc18101620 2023-12-17T12:10:06.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.769+0000 7fcc157fa700 1 -- 172.21.15.27:0/1220977460 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) v1 ==== 56+0+5285 (secure 0 0 0) 0x7fcc080996a0 con 0x7fcc18101620 2023-12-17T12:10:06.770 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 84 2023-12-17T12:10:06.770 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:created 2023-12-17T12:01:24.596523+0000 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2023-12-17T12:09:24.507787+0000 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 26 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client luminous 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release reef 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2023-12-17T12:10:06.771 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6802/4043808604,v1:172.21.15.27:6803/4043808604] [v2:172.21.15.27:6804/4043808604,v1:172.21.15.27:6805/4043808604] exists,up 3fb8ca94-6c9e-452e-a99f-3b84eb305dc1 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.141:6800/2130638652,v1:172.21.15.141:6801/2130638652] [v2:172.21.15.141:6802/2130638652,v1:172.21.15.141:6803/2130638652] exists,up 5f6fafe1-a95f-4916-9c2f-545e8be564ae 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/3357629736,v1:172.21.15.192:6801/3357629736] [v2:172.21.15.192:6802/3357629736,v1:172.21.15.192:6803/3357629736] exists,up 35099caf-f95c-4657-8db5-ebbfb1c88fb2 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 25 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6810/130987497,v1:172.21.15.27:6811/130987497] [v2:172.21.15.27:6812/130987497,v1:172.21.15.27:6813/130987497] exists,up 9aed4e22-de72-4ad5-8b0d-0c5d159c7152 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 32 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.141:6808/2292839453,v1:172.21.15.141:6809/2292839453] [v2:172.21.15.141:6810/2292839453,v1:172.21.15.141:6811/2292839453] exists,up 0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 37 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/1558095856,v1:172.21.15.192:6809/1558095856] [v2:172.21.15.192:6810/1558095856,v1:172.21.15.192:6811/1558095856] exists,up ee59d302-1490-4e7e-ab79-af6243c4e97b 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 44 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6818/3435247687,v1:172.21.15.27:6819/3435247687] [v2:172.21.15.27:6820/3435247687,v1:172.21.15.27:6821/3435247687] exists,up 5a2aa985-c312-47be-b27a-59be5ce1ab30 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 49 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.141:6816/303714456,v1:172.21.15.141:6817/303714456] [v2:172.21.15.141:6818/303714456,v1:172.21.15.141:6819/303714456] exists,up bdadd696-7e4a-411e-8366-17d9cb235d00 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 56 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/1098230082,v1:172.21.15.192:6817/1098230082] [v2:172.21.15.192:6818/1098230082,v1:172.21.15.192:6819/1098230082] exists,up 92586b7e-b4c6-4e69-aecb-e7696a821a2f 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 64 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6826/3328001767,v1:172.21.15.27:6827/3328001767] [v2:172.21.15.27:6828/3328001767,v1:172.21.15.27:6829/3328001767] exists,up c11178c6-bb54-4043-a57c-e5c22a1560ef 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 69 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.141:6824/2486275006,v1:172.21.15.141:6825/2486275006] [v2:172.21.15.141:6826/2486275006,v1:172.21.15.141:6827/2486275006] exists,up 2916e700-28e1-4b61-aedb-55dbd8b44208 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 77 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/2290390540,v1:172.21.15.192:6825/2290390540] [v2:172.21.15.192:6826/2290390540,v1:172.21.15.192:6827/2290390540] exists,up c9e69f0d-0fe3-4559-93ac-a66f20c0ce15 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.6 [8,11] 2023-12-17T12:10:06.772 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.b [2,4] 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.14 [2,4] 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.18 [5,11] 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.1f [8,1] 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.c [3,9] 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.13 [4,9] 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.24 [4,9] 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/674094332 expires 2023-12-18T12:02:22.716630+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/674094332 expires 2023-12-18T12:02:22.716630+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2021030429 expires 2023-12-18T12:01:54.913305+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1319914711 expires 2023-12-18T12:02:22.716630+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/1943739335 expires 2023-12-18T12:01:54.913305+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/166661733 expires 2023-12-18T12:02:22.716630+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2644173181 expires 2023-12-18T12:01:54.913305+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/1943739335 expires 2023-12-18T12:01:54.913305+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1804386639 expires 2023-12-18T12:01:54.913305+0000 2023-12-17T12:10:06.773 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/4225064018 expires 2023-12-18T12:02:22.716630+0000 2023-12-17T12:10:06.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcc000a1510 msgr2=0x7fcc000a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:06.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcc000a1510 0x7fcc000a39d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00b580 tx=0x7fcc0c005e80 comp rx=0 tx=0).stop 2023-12-17T12:10:06.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18101620 msgr2=0x7fcc181996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:06.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18101620 0x7fcc181996a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800d9b0 tx=0x7fcc0800dcc0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 shutdown_connections 2023-12-17T12:10:06.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fcc000a1510 0x7fcc000a39d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc181031c0 0x7fcc181a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc18101620 0x7fcc181996a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 --2- 172.21.15.27:0/1220977460 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fcc18102820 0x7fcc18199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:06.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.771+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 >> 172.21.15.27:0/1220977460 conn(0x7fcc180fcbe0 msgr2=0x7fcc18108090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:06.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.772+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 shutdown_connections 2023-12-17T12:10:06.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:06.772+0000 7fcc1e411700 1 -- 172.21.15.27:0/1220977460 wait complete. 2023-12-17T12:10:06.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:06 smithi027 ceph-mon[36871]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:10:07.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:06 smithi141 ceph-mon[41225]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:10:07.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:06 smithi192 ceph-mon[41899]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-17T12:10:07.218 INFO:teuthology.run_tasks:Running task fs.ready... 2023-12-17T12:10:07.226 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 82, in run_one_task task = get_task(taskname) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 35, in get_task module = _import('tasks', module_name, task_name, fail_on_import_error=True) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 54, in _import module = __import__( ModuleNotFoundError: No module named 'tasks.fs' 2023-12-17T12:10:07.287 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=863a38a57a23417babca9d1a66fe6826 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 82, in run_one_task task = get_task(taskname) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 35, in get_task module = _import('tasks', module_name, task_name, fail_on_import_error=True) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 54, in _import module = __import__( ModuleNotFoundError: No module named 'tasks.fs' 2023-12-17T12:10:07.288 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-17T12:10:07.296 INFO:tasks.cephadm:Teardown begin 2023-12-17T12:10:07.297 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-17T12:10:07.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:10:07.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 -- 172.21.15.27:0/2708122534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c102690 msgr2=0x7faa9c102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:07.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 --2- 172.21.15.27:0/2708122534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c102690 0x7faa9c102af0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faa840099c0 tx=0x7faa84009cd0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 -- 172.21.15.27:0/2708122534 shutdown_connections 2023-12-17T12:10:07.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 --2- 172.21.15.27:0/2708122534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa9c103030 0x7faa9c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 --2- 172.21.15.27:0/2708122534 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7faa9c1014a0 0x7faa9c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 --2- 172.21.15.27:0/2708122534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c102690 0x7faa9c102af0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 -- 172.21.15.27:0/2708122534 >> 172.21.15.27:0/2708122534 conn(0x7faa9c0fca10 msgr2=0x7faa9c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:07.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 -- 172.21.15.27:0/2708122534 shutdown_connections 2023-12-17T12:10:07.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.511+0000 7faaa1fff700 1 -- 172.21.15.27:0/2708122534 wait complete. 2023-12-17T12:10:07.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.512+0000 7faaa1fff700 1 Processor -- start 2023-12-17T12:10:07.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.512+0000 7faaa1fff700 1 -- start start 2023-12-17T12:10:07.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faaa1fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa9c1014a0 0x7faa9c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:07.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faaa1fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7faa9c102690 0x7faa9c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:07.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faaa1fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c103030 0x7faa9c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:07.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faaa1fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1a1070 con 0x7faa9c103030 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faaa1fff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1a11b0 con 0x7faa9c102690 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faaa1fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1a1340 con 0x7faa9c1014a0 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faa9bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c103030 0x7faa9c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faa9bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c103030 0x7faa9c1a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42970/0 (socket says 172.21.15.27:42970) 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faa9bfff700 1 -- 172.21.15.27:0/1100843058 learned_addr learned my addr 172.21.15.27:0/1100843058 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faa9b7fe700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa9c1014a0 0x7faa9c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.513+0000 7faa93fff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7faa9c102690 0x7faa9c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa9bfff700 1 -- 172.21.15.27:0/1100843058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa9c1014a0 msgr2=0x7faa9c199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:07.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa9bfff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa9c1014a0 0x7faa9c199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa9bfff700 1 -- 172.21.15.27:0/1100843058 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7faa9c102690 msgr2=0x7faa9c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa9bfff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7faa9c102690 0x7faa9c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa9bfff700 1 -- 172.21.15.27:0/1100843058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa840095f0 con 0x7faa9c103030 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa93fff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7faa9c102690 0x7faa9c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa9bfff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c103030 0x7faa9c1a0aa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faa94007ee0 tx=0x7faa9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa997fa700 1 -- 172.21.15.27:0/1100843058 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94017070 con 0x7faa9c103030 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa997fa700 1 -- 172.21.15.27:0/1100843058 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa9400f040 con 0x7faa9c103030 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9c1a1620 con 0x7faa9c103030 2023-12-17T12:10:07.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faa997fa700 1 -- 172.21.15.27:0/1100843058 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94005710 con 0x7faa9c103030 2023-12-17T12:10:07.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.514+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c1a1ae0 con 0x7faa9c103030 2023-12-17T12:10:07.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.515+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9c068230 con 0x7faa9c103030 2023-12-17T12:10:07.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.517+0000 7faa997fa700 1 -- 172.21.15.27:0/1100843058 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7faa94005870 con 0x7faa9c103030 2023-12-17T12:10:07.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.519+0000 7faa997fa700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faa880a1510 0x7faa880a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:07.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.519+0000 7faa997fa700 1 -- 172.21.15.27:0/1100843058 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faa940cc6f0 con 0x7faa9c103030 2023-12-17T12:10:07.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.519+0000 7faa9b7fe700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faa880a1510 0x7faa880a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:07.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.520+0000 7faa9b7fe700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faa880a1510 0x7faa880a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa9c1024f0 tx=0x7faa8c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:07.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.523+0000 7faa997fa700 1 -- 172.21.15.27:0/1100843058 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faa94095ba0 con 0x7faa9c103030 2023-12-17T12:10:07.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:07 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1220977460' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-17T12:10:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.784+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa9c10cbc0 con 0x7faa9c103030 2023-12-17T12:10:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.784+0000 7faa997fa700 1 -- 172.21.15.27:0/1100843058 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11327 (secure 0 0 0) 0x7faa9c10cbc0 con 0x7faa9c103030 2023-12-17T12:10:07.786 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:10:07.786 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3551652066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3551652066},{"type":"v1","addr":"172.21.15.27:6841","nonce":3551652066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-17T12:08:48.917098+0000","modified":"2023-12-17T12:08:55.420903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24538,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.141:6839/3282544216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6838","nonce":3282544216},{"type":"v1","addr":"172.21.15.141:6839","nonce":3282544216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-17T12:10:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faa880a1510 msgr2=0x7faa880a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faa880a1510 0x7faa880a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa9c1024f0 tx=0x7faa8c009260 comp rx=0 tx=0).stop 2023-12-17T12:10:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c103030 msgr2=0x7faa9c1a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c103030 0x7faa9c1a0aa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faa94007ee0 tx=0x7faa9400d3b0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 shutdown_connections 2023-12-17T12:10:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faa880a1510 0x7faa880a39d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa9c1014a0 0x7faa9c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa9c103030 0x7faa9c1a0aa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 --2- 172.21.15.27:0/1100843058 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7faa9c102690 0x7faa9c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 >> 172.21.15.27:0/1100843058 conn(0x7faa9c0fca10 msgr2=0x7faa9c107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 shutdown_connections 2023-12-17T12:10:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:07.787+0000 7faaa1fff700 1 -- 172.21.15.27:0/1100843058 wait complete. 2023-12-17T12:10:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-17T12:10:07.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-17T12:10:08.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:07 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1220977460' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-17T12:10:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.015+0000 7fab065ff700 1 -- 172.21.15.27:0/2319751778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 msgr2=0x7fab0010b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.015+0000 7fab065ff700 1 --2- 172.21.15.27:0/2319751778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 0x7fab0010b7f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faaf800b0a0 tx=0x7faaf800b3b0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.015+0000 7fab065ff700 1 -- 172.21.15.27:0/2319751778 shutdown_connections 2023-12-17T12:10:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.015+0000 7fab065ff700 1 --2- 172.21.15.27:0/2319751778 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab000755b0 0x7fab00073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.015+0000 7fab065ff700 1 --2- 172.21.15.27:0/2319751778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 0x7fab0010b7f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.015+0000 7fab065ff700 1 --2- 172.21.15.27:0/2319751778 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab00073fd0 0x7fab00074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.015+0000 7fab065ff700 1 -- 172.21.15.27:0/2319751778 >> 172.21.15.27:0/2319751778 conn(0x7fab000fca20 msgr2=0x7fab000fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:08.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.016+0000 7fab065ff700 1 -- 172.21.15.27:0/2319751778 shutdown_connections 2023-12-17T12:10:08.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.016+0000 7fab065ff700 1 -- 172.21.15.27:0/2319751778 wait complete. 2023-12-17T12:10:08.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab065ff700 1 Processor -- start 2023-12-17T12:10:08.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab065ff700 1 -- start start 2023-12-17T12:10:08.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab065ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab00073fd0 0x7fab001995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab065ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab000755b0 0x7fab00199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab065ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 0x7fab001a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab065ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab001a1100 con 0x7fab00109380 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab065ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab001a1240 con 0x7fab00073fd0 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab065ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab001a13d0 con 0x7fab000755b0 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab04b9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 0x7fab001a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab04b9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 0x7fab001a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42982/0 (socket says 172.21.15.27:42982) 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7fab04b9c700 1 -- 172.21.15.27:0/2350548118 learned_addr learned my addr 172.21.15.27:0/2350548118 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7faaff7fe700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab000755b0 0x7fab00199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.017+0000 7faafffff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab00073fd0 0x7fab001995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7fab04b9c700 1 -- 172.21.15.27:0/2350548118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab000755b0 msgr2=0x7fab00199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7fab04b9c700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab000755b0 0x7fab00199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7fab04b9c700 1 -- 172.21.15.27:0/2350548118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab00073fd0 msgr2=0x7fab001995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7fab04b9c700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab00073fd0 0x7fab001995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7fab04b9c700 1 -- 172.21.15.27:0/2350548118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8009d00 con 0x7fab00109380 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7fab04b9c700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 0x7fab001a0b30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faaf800b780 tx=0x7faaf800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7faafd7fa700 1 -- 172.21.15.27:0/2350548118 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf800e040 con 0x7fab00109380 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab001a1650 con 0x7fab00109380 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7faafd7fa700 1 -- 172.21.15.27:0/2350548118 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaf8003c00 con 0x7fab00109380 2023-12-17T12:10:08.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7faafd7fa700 1 -- 172.21.15.27:0/2350548118 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf80052a0 con 0x7fab00109380 2023-12-17T12:10:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.018+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab001a1a80 con 0x7fab00109380 2023-12-17T12:10:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.019+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab00108190 con 0x7fab00109380 2023-12-17T12:10:08.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.021+0000 7faafd7fa700 1 -- 172.21.15.27:0/2350548118 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7faaf801d480 con 0x7fab00109380 2023-12-17T12:10:08.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.023+0000 7faafd7fa700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faae80a1510 0x7faae80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.023+0000 7faafd7fa700 1 -- 172.21.15.27:0/2350548118 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faaf80cde20 con 0x7fab00109380 2023-12-17T12:10:08.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.023+0000 7faafffff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faae80a1510 0x7faae80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.024+0000 7faafffff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faae80a1510 0x7faae80a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009c60 tx=0x7faaf0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.027+0000 7faafd7fa700 1 -- 172.21.15.27:0/2350548118 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faaf80973c0 con 0x7fab00109380 2023-12-17T12:10:08.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:07 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1220977460' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-17T12:10:08.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.287+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab00068230 con 0x7fab00109380 2023-12-17T12:10:08.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.287+0000 7faafd7fa700 1 -- 172.21.15.27:0/2350548118 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11327 (secure 0 0 0) 0x7faaf809a9e0 con 0x7fab00109380 2023-12-17T12:10:08.289 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:10:08.289 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2608720361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2608720361},{"type":"v1","addr":"172.21.15.27:6837","nonce":2608720361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/1099412442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":1099412442},{"type":"v1","addr":"172.21.15.27:6839","nonce":1099412442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3551652066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3551652066},{"type":"v1","addr":"172.21.15.27:6841","nonce":3551652066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2710858125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2710858125},{"type":"v1","addr":"172.21.15.192:6833","nonce":2710858125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6833/2577446255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6832","nonce":2577446255},{"type":"v1","addr":"172.21.15.141:6833","nonce":2577446255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2531226190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2531226190},{"type":"v1","addr":"172.21.15.27:6835","nonce":2531226190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-17T12:08:48.917098+0000","modified":"2023-12-17T12:08:55.420903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24538,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.141:6835/4280622893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":4280622893},{"type":"v1","addr":"172.21.15.141:6835","nonce":4280622893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/2634479037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2634479037},{"type":"v1","addr":"172.21.15.192:6835","nonce":2634479037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/1847164381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1847164381},{"type":"v1","addr":"172.21.15.192:6837","nonce":1847164381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.141:6837/3627913634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6836","nonce":3627913634},{"type":"v1","addr":"172.21.15.141:6837","nonce":3627913634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/962693048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":962693048},{"type":"v1","addr":"172.21.15.192:6839","nonce":962693048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.141:6839/3282544216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6838","nonce":3282544216},{"type":"v1","addr":"172.21.15.141:6839","nonce":3282544216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-17T12:10:08.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faae80a1510 msgr2=0x7faae80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faae80a1510 0x7faae80a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009c60 tx=0x7faaf0009260 comp rx=0 tx=0).stop 2023-12-17T12:10:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 msgr2=0x7fab001a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 0x7fab001a0b30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faaf800b780 tx=0x7faaf800bab0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 shutdown_connections 2023-12-17T12:10:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7faae80a1510 0x7faae80a39d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab00109380 0x7fab001a0b30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fab00073fd0 0x7fab001995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 --2- 172.21.15.27:0/2350548118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab000755b0 0x7fab00199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 >> 172.21.15.27:0/2350548118 conn(0x7fab000fca20 msgr2=0x7fab001052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 shutdown_connections 2023-12-17T12:10:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.290+0000 7fab065ff700 1 -- 172.21.15.27:0/2350548118 wait complete. 2023-12-17T12:10:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-17T12:10:08.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-17T12:10:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 -- 172.21.15.27:0/1644661725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c1014a0 msgr2=0x7f8e9c1018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/1644661725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c1014a0 0x7f8e9c1018c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0099c0 tx=0x7f8e8c009cd0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 -- 172.21.15.27:0/1644661725 shutdown_connections 2023-12-17T12:10:08.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/1644661725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c103030 0x7f8e9c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/1644661725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c1014a0 0x7f8e9c1018c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/1644661725 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8e9c102690 0x7f8e9c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 -- 172.21.15.27:0/1644661725 >> 172.21.15.27:0/1644661725 conn(0x7f8e9c0fca10 msgr2=0x7f8e9c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:08.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 -- 172.21.15.27:0/1644661725 shutdown_connections 2023-12-17T12:10:08.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.519+0000 7f8ea0fff700 1 -- 172.21.15.27:0/1644661725 wait complete. 2023-12-17T12:10:08.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.520+0000 7f8ea0fff700 1 Processor -- start 2023-12-17T12:10:08.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.520+0000 7f8ea0fff700 1 -- start start 2023-12-17T12:10:08.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8ea0fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8e9c1014a0 0x7f8e9c1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8ea0fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c102690 0x7f8e9c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8ea0fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c103030 0x7f8e9c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8ea0fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1a1020 con 0x7f8e9c102690 2023-12-17T12:10:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8ea0fff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1a1160 con 0x7f8e9c1014a0 2023-12-17T12:10:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8ea0fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1a12f0 con 0x7f8e9c103030 2023-12-17T12:10:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e99d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c102690 0x7f8e9c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e99d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c102690 0x7f8e9c199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42998/0 (socket says 172.21.15.27:42998) 2023-12-17T12:10:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e99d9b700 1 -- 172.21.15.27:0/2285805041 learned_addr learned my addr 172.21.15.27:0/2285805041 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e9ad9d700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c103030 0x7f8e9c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e9a59c700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8e9c1014a0 0x7f8e9c1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e99d9b700 1 -- 172.21.15.27:0/2285805041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c103030 msgr2=0x7f8e9c1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e99d9b700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c103030 0x7f8e9c1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e99d9b700 1 -- 172.21.15.27:0/2285805041 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8e9c1014a0 msgr2=0x7f8e9c1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e99d9b700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8e9c1014a0 0x7f8e9c1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.521+0000 7f8e99d9b700 1 -- 172.21.15.27:0/2285805041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0095f0 con 0x7f8e9c102690 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.522+0000 7f8e99d9b700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c102690 0x7f8e9c199a30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000c9b0 tx=0x7f8e9000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.522+0000 7f8e8b7fe700 1 -- 172.21.15.27:0/2285805041 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90007b00 con 0x7f8e9c102690 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.522+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c1a15d0 con 0x7f8e9c102690 2023-12-17T12:10:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.522+0000 7f8e8b7fe700 1 -- 172.21.15.27:0/2285805041 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e90007c60 con 0x7f8e9c102690 2023-12-17T12:10:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.522+0000 7f8e8b7fe700 1 -- 172.21.15.27:0/2285805041 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90018660 con 0x7f8e9c102690 2023-12-17T12:10:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.522+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c1a1ae0 con 0x7f8e9c102690 2023-12-17T12:10:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.523+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9c068230 con 0x7f8e9c102690 2023-12-17T12:10:08.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.525+0000 7f8e8b7fe700 1 -- 172.21.15.27:0/2285805041 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f8e9000f450 con 0x7f8e9c102690 2023-12-17T12:10:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.526+0000 7f8e8b7fe700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8e840a1510 0x7f8e840a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.526+0000 7f8e8b7fe700 1 -- 172.21.15.27:0/2285805041 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8e900ccee0 con 0x7f8e9c102690 2023-12-17T12:10:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.527+0000 7f8e9a59c700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8e840a1510 0x7f8e840a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.527+0000 7f8e9a59c700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8e840a1510 0x7f8e840a39d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000c00 tx=0x7f8e8c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.530+0000 7f8e8b7fe700 1 -- 172.21.15.27:0/2285805041 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8e90096390 con 0x7f8e9c102690 2023-12-17T12:10:08.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.745+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e9c10cbb0 con 0x7f8e9c102690 2023-12-17T12:10:08.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.747+0000 7f8e8b7fe700 1 -- 172.21.15.27:0/2285805041 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22507 (secure 0 0 0) 0x7f8e9000fe60 con 0x7f8e9c102690 2023-12-17T12:10:08.748 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-17T12:10:08.748 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"62030760-9cd3-11ee-95a5-87774f69a715","created":"2023-12-17T12:01:24.596523+0000","modified":"2023-12-17T12:09:24.507787+0000","last_up_change":"2023-12-17T12:08:02.034092+0000","last_in_change":"2023-12-17T12:07:43.711944+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-17T12:04:41.530587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-17T12:08:47.025470+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-17T12:08:48.366216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3fb8ca94-6c9e-452e-a99f-3b84eb305dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6803","nonce":4043808604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6805","nonce":4043808604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6809","nonce":4043808604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4043808604},{"type":"v1","addr":"172.21.15.27:6807","nonce":4043808604}]},"public_addr":"172.21.15.27:6803/4043808604","cluster_addr":"172.21.15.27:6805/4043808604","heartbeat_back_addr":"172.21.15.27:6809/4043808604","heartbeat_front_addr":"172.21.15.27:6807/4043808604","state":["exists","up"]},{"osd":1,"uuid":"5f6fafe1-a95f-4916-9c2f-545e8be564ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6801","nonce":2130638652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6803","nonce":2130638652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6807","nonce":2130638652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":2130638652},{"type":"v1","addr":"172.21.15.141:6805","nonce":2130638652}]},"public_addr":"172.21.15.141:6801/2130638652","cluster_addr":"172.21.15.141:6803/2130638652","heartbeat_back_addr":"172.21.15.141:6807/2130638652","heartbeat_front_addr":"172.21.15.141:6805/2130638652","state":["exists","up"]},{"osd":2,"uuid":"35099caf-f95c-4657-8db5-ebbfb1c88fb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6801","nonce":3357629736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6803","nonce":3357629736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6807","nonce":3357629736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3357629736},{"type":"v1","addr":"172.21.15.192:6805","nonce":3357629736}]},"public_addr":"172.21.15.192:6801/3357629736","cluster_addr":"172.21.15.192:6803/3357629736","heartbeat_back_addr":"172.21.15.192:6807/3357629736","heartbeat_front_addr":"172.21.15.192:6805/3357629736","state":["exists","up"]},{"osd":3,"uuid":"9aed4e22-de72-4ad5-8b0d-0c5d159c7152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6811","nonce":130987497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6813","nonce":130987497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6817","nonce":130987497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":130987497},{"type":"v1","addr":"172.21.15.27:6815","nonce":130987497}]},"public_addr":"172.21.15.27:6811/130987497","cluster_addr":"172.21.15.27:6813/130987497","heartbeat_back_addr":"172.21.15.27:6817/130987497","heartbeat_front_addr":"172.21.15.27:6815/130987497","state":["exists","up"]},{"osd":4,"uuid":"0e8c67b4-4777-46c9-b7a3-12ffc26e1bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6809","nonce":2292839453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6811","nonce":2292839453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6815","nonce":2292839453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":2292839453},{"type":"v1","addr":"172.21.15.141:6813","nonce":2292839453}]},"public_addr":"172.21.15.141:6809/2292839453","cluster_addr":"172.21.15.141:6811/2292839453","heartbeat_back_addr":"172.21.15.141:6815/2292839453","heartbeat_front_addr":"172.21.15.141:6813/2292839453","state":["exists","up"]},{"osd":5,"uuid":"ee59d302-1490-4e7e-ab79-af6243c4e97b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1558095856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1558095856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1558095856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1558095856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1558095856}]},"public_addr":"172.21.15.192:6809/1558095856","cluster_addr":"172.21.15.192:6811/1558095856","heartbeat_back_addr":"172.21.15.192:6815/1558095856","heartbeat_front_addr":"172.21.15.192:6813/1558095856","state":["exists","up"]},{"osd":6,"uuid":"5a2aa985-c312-47be-b27a-59be5ce1ab30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6819","nonce":3435247687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6821","nonce":3435247687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6825","nonce":3435247687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3435247687},{"type":"v1","addr":"172.21.15.27:6823","nonce":3435247687}]},"public_addr":"172.21.15.27:6819/3435247687","cluster_addr":"172.21.15.27:6821/3435247687","heartbeat_back_addr":"172.21.15.27:6825/3435247687","heartbeat_front_addr":"172.21.15.27:6823/3435247687","state":["exists","up"]},{"osd":7,"uuid":"bdadd696-7e4a-411e-8366-17d9cb235d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6817","nonce":303714456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6819","nonce":303714456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6822","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6823","nonce":303714456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":303714456},{"type":"v1","addr":"172.21.15.141:6821","nonce":303714456}]},"public_addr":"172.21.15.141:6817/303714456","cluster_addr":"172.21.15.141:6819/303714456","heartbeat_back_addr":"172.21.15.141:6823/303714456","heartbeat_front_addr":"172.21.15.141:6821/303714456","state":["exists","up"]},{"osd":8,"uuid":"92586b7e-b4c6-4e69-aecb-e7696a821a2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6817","nonce":1098230082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6819","nonce":1098230082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6823","nonce":1098230082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1098230082},{"type":"v1","addr":"172.21.15.192:6821","nonce":1098230082}]},"public_addr":"172.21.15.192:6817/1098230082","cluster_addr":"172.21.15.192:6819/1098230082","heartbeat_back_addr":"172.21.15.192:6823/1098230082","heartbeat_front_addr":"172.21.15.192:6821/1098230082","state":["exists","up"]},{"osd":9,"uuid":"c11178c6-bb54-4043-a57c-e5c22a1560ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6827","nonce":3328001767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6829","nonce":3328001767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6833","nonce":3328001767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3328001767},{"type":"v1","addr":"172.21.15.27:6831","nonce":3328001767}]},"public_addr":"172.21.15.27:6827/3328001767","cluster_addr":"172.21.15.27:6829/3328001767","heartbeat_back_addr":"172.21.15.27:6833/3328001767","heartbeat_front_addr":"172.21.15.27:6831/3328001767","state":["exists","up"]},{"osd":10,"uuid":"2916e700-28e1-4b61-aedb-55dbd8b44208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6825","nonce":2486275006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6826","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6827","nonce":2486275006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6831","nonce":2486275006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":2486275006},{"type":"v1","addr":"172.21.15.141:6829","nonce":2486275006}]},"public_addr":"172.21.15.141:6825/2486275006","cluster_addr":"172.21.15.141:6827/2486275006","heartbeat_back_addr":"172.21.15.141:6831/2486275006","heartbeat_front_addr":"172.21.15.141:6829/2486275006","state":["exists","up"]},{"osd":11,"uuid":"c9e69f0d-0fe3-4559-93ac-a66f20c0ce15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6825","nonce":2290390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6827","nonce":2290390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6831","nonce":2290390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2290390540},{"type":"v1","addr":"172.21.15.192:6829","nonce":2290390540}]},"public_addr":"172.21.15.192:6825/2290390540","cluster_addr":"172.21.15.192:6827/2290390540","heartbeat_back_addr":"172.21.15.192:6831/2290390540","heartbeat_front_addr":"172.21.15.192:6829/2290390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:03:54.222110+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:16.160789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:04:38.469331+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:00.997915+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:23.310612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:05:45.692115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:08.416038+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:30.539430+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:06:52.906427+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:15.824072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:07:38.253148+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-17T12:08:00.777315+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/674094332":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2021030429":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1319914711":"2023-12-18T12:02:22.716630+0000","172.21.15.27:6800/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/166661733":"2023-12-18T12:02:22.716630+0000","172.21.15.27:0/2644173181":"2023-12-18T12:01:54.913305+0000","172.21.15.27:6801/1943739335":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/1804386639":"2023-12-18T12:01:54.913305+0000","172.21.15.27:0/4225064018":"2023-12-18T12:02:22.716630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-17T12:10:08.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.749+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8e840a1510 msgr2=0x7f8e840a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.749+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8e840a1510 0x7f8e840a39d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000c00 tx=0x7f8e8c005fb0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.749+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c102690 msgr2=0x7f8e9c199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.749+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c102690 0x7f8e9c199a30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000c9b0 tx=0x7f8e9000ccc0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.750+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 shutdown_connections 2023-12-17T12:10:08.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.750+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f8e840a1510 0x7f8e840a39d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.750+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c103030 0x7f8e9c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.750+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8e9c1014a0 0x7f8e9c1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.750+0000 7f8ea0fff700 1 --2- 172.21.15.27:0/2285805041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e9c102690 0x7f8e9c199a30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.750+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 >> 172.21.15.27:0/2285805041 conn(0x7f8e9c0fca10 msgr2=0x7f8e9c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.750+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 shutdown_connections 2023-12-17T12:10:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.750+0000 7f8ea0fff700 1 -- 172.21.15.27:0/2285805041 wait complete. 2023-12-17T12:10:08.770 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-17T12:10:08.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:08 smithi027 ceph-mon[36871]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:10:08.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:08 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1100843058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:10:08.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:08 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2350548118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:10:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 -- 172.21.15.27:0/758859861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e30102790 msgr2=0x7f4e30102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 --2- 172.21.15.27:0/758859861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e30102790 0x7f4e30102bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e240099c0 tx=0x7f4e24009cd0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 -- 172.21.15.27:0/758859861 shutdown_connections 2023-12-17T12:10:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 --2- 172.21.15.27:0/758859861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e30103130 0x7f4e301099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 --2- 172.21.15.27:0/758859861 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e301015a0 0x7f4e301019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 --2- 172.21.15.27:0/758859861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e30102790 0x7f4e30102bf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 -- 172.21.15.27:0/758859861 >> 172.21.15.27:0/758859861 conn(0x7f4e300fcb10 msgr2=0x7f4e300fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:08.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 -- 172.21.15.27:0/758859861 shutdown_connections 2023-12-17T12:10:08.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.979+0000 7f4e365ff700 1 -- 172.21.15.27:0/758859861 wait complete. 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.980+0000 7f4e365ff700 1 Processor -- start 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e365ff700 1 -- start start 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e365ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e301015a0 0x7f4e30197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e365ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e30102790 0x7f4e30197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e365ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e30103130 0x7f4e3019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e365ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3019ef50 con 0x7f4e301015a0 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e365ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3019f090 con 0x7f4e30102790 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e365ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3019f220 con 0x7f4e30103130 2023-12-17T12:10:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e2ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e301015a0 0x7f4e30197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e2ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e301015a0 0x7f4e30197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43024/0 (socket says 172.21.15.27:43024) 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e2ffff700 1 -- 172.21.15.27:0/2658909450 learned_addr learned my addr 172.21.15.27:0/2658909450 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e2f7fe700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e30102790 0x7f4e30197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.981+0000 7f4e34b9c700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e30103130 0x7f4e3019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2ffff700 1 -- 172.21.15.27:0/2658909450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e30103130 msgr2=0x7f4e3019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2ffff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e30103130 0x7f4e3019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2ffff700 1 -- 172.21.15.27:0/2658909450 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e30102790 msgr2=0x7f4e30197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2ffff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e30102790 0x7f4e30197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2ffff700 1 -- 172.21.15.27:0/2658909450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240095f0 con 0x7f4e301015a0 2023-12-17T12:10:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e34b9c700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e30103130 0x7f4e3019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:10:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2ffff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e301015a0 0x7f4e30197420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000d980 tx=0x7f4e2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2d7fa700 1 -- 172.21.15.27:0/2658909450 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20004020 con 0x7f4e301015a0 2023-12-17T12:10:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2d7fa700 1 -- 172.21.15.27:0/2658909450 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e20009730 con 0x7f4e301015a0 2023-12-17T12:10:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3019f500 con 0x7f4e301015a0 2023-12-17T12:10:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e2d7fa700 1 -- 172.21.15.27:0/2658909450 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20003a00 con 0x7f4e301015a0 2023-12-17T12:10:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.982+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3019fa60 con 0x7f4e301015a0 2023-12-17T12:10:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.984+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e30108a70 con 0x7f4e301015a0 2023-12-17T12:10:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.985+0000 7f4e2d7fa700 1 -- 172.21.15.27:0/2658909450 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f4e200098a0 con 0x7f4e301015a0 2023-12-17T12:10:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.987+0000 7f4e2d7fa700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e180a1590 0x7f4e180a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.987+0000 7f4e2d7fa700 1 -- 172.21.15.27:0/2658909450 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4e200ccb50 con 0x7f4e301015a0 2023-12-17T12:10:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.987+0000 7f4e2f7fe700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e180a1590 0x7f4e180a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.988+0000 7f4e2f7fe700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e180a1590 0x7f4e180a3a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009990 tx=0x7f4e24000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:08.991+0000 7f4e2d7fa700 1 -- 172.21.15.27:0/2658909450 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4e20096000 con 0x7f4e301015a0 2023-12-17T12:10:09.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:08 smithi141 ceph-mon[41225]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:10:09.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:08 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1100843058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:10:09.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:08 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2350548118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:10:09.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:08 smithi192 ceph-mon[41899]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-17T12:10:09.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:08 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1100843058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:10:09.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:08 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2350548118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-17T12:10:09.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.260+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4e30068310 con 0x7f4e301015a0 2023-12-17T12:10:09.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.262+0000 7f4e2d7fa700 1 -- 172.21.15.27:0/2658909450 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4e20099620 con 0x7f4e301015a0 2023-12-17T12:10:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e180a1590 msgr2=0x7f4e180a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e180a1590 0x7f4e180a3a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009990 tx=0x7f4e24000bc0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e301015a0 msgr2=0x7f4e30197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e301015a0 0x7f4e30197420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000d980 tx=0x7f4e2000dc90 comp rx=0 tx=0).stop 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 shutdown_connections 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f4e180a1590 0x7f4e180a3a50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e30103130 0x7f4e3019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e301015a0 0x7f4e30197420 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 --2- 172.21.15.27:0/2658909450 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4e30102790 0x7f4e30197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.265+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 >> 172.21.15.27:0/2658909450 conn(0x7f4e300fcb10 msgr2=0x7f4e30105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:09.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.266+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 shutdown_connections 2023-12-17T12:10:09.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.266+0000 7f4e365ff700 1 -- 172.21.15.27:0/2658909450 wait complete. 2023-12-17T12:10:09.268 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-17T12:10:09.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.496+0000 7fa18f7ff700 1 -- 172.21.15.27:0/629644353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 msgr2=0x7fa188109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.496+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/629644353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 0x7fa188109710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa180009990 tx=0x7fa180009ca0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.496+0000 7fa18f7ff700 1 -- 172.21.15.27:0/629644353 shutdown_connections 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.496+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/629644353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 0x7fa188109710 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.496+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/629644353 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa1881020c0 0x7fa1881044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.496+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/629644353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1881049f0 0x7fa188106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.496+0000 7fa18f7ff700 1 -- 172.21.15.27:0/629644353 >> 172.21.15.27:0/629644353 conn(0x7fa1880fb9f0 msgr2=0x7fa1880fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.496+0000 7fa18f7ff700 1 -- 172.21.15.27:0/629644353 shutdown_connections 2023-12-17T12:10:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.497+0000 7fa18f7ff700 1 -- 172.21.15.27:0/629644353 wait complete. 2023-12-17T12:10:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.497+0000 7fa18f7ff700 1 Processor -- start 2023-12-17T12:10:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18f7ff700 1 -- start start 2023-12-17T12:10:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18f7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1881020c0 0x7fa1881019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18f7ff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa1881049f0 0x7fa188100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18f7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 0x7fa1881005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18f7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa188100d50 con 0x7fa188107320 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18f7ff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa188100ec0 con 0x7fa1881049f0 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18f7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1881a1440 con 0x7fa1881020c0 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18dd9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 0x7fa1881005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18dd9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 0x7fa1881005b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43044/0 (socket says 172.21.15.27:43044) 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18dd9c700 1 -- 172.21.15.27:0/1467195844 learned_addr learned my addr 172.21.15.27:0/1467195844 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.498+0000 7fa18cd9a700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa1881049f0 0x7fa188100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18d59b700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1881020c0 0x7fa1881019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18dd9c700 1 -- 172.21.15.27:0/1467195844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1881020c0 msgr2=0x7fa1881019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18dd9c700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1881020c0 0x7fa1881019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18dd9c700 1 -- 172.21.15.27:0/1467195844 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa1881049f0 msgr2=0x7fa188100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18dd9c700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa1881049f0 0x7fa188100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18dd9c700 1 -- 172.21.15.27:0/1467195844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1800095f0 con 0x7fa188107320 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18cd9a700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa1881049f0 0x7fa188100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18dd9c700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 0x7fa1881005b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa180005910 tx=0x7fa1800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa17e7fc700 1 -- 172.21.15.27:0/1467195844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18001d070 con 0x7fa188107320 2023-12-17T12:10:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa17e7fc700 1 -- 172.21.15.27:0/1467195844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa18000be30 con 0x7fa188107320 2023-12-17T12:10:09.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa17e7fc700 1 -- 172.21.15.27:0/1467195844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18000f460 con 0x7fa188107320 2023-12-17T12:10:09.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1881a1670 con 0x7fa188107320 2023-12-17T12:10:09.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.499+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1881a1a30 con 0x7fa188107320 2023-12-17T12:10:09.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.501+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1880fd570 con 0x7fa188107320 2023-12-17T12:10:09.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.503+0000 7fa17e7fc700 1 -- 172.21.15.27:0/1467195844 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa18000b9a0 con 0x7fa188107320 2023-12-17T12:10:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.505+0000 7fa17e7fc700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa1740a1590 0x7fa1740a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.505+0000 7fa17e7fc700 1 -- 172.21.15.27:0/1467195844 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa1800ce570 con 0x7fa188107320 2023-12-17T12:10:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.505+0000 7fa18d59b700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa1740a1590 0x7fa1740a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.506+0000 7fa18d59b700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa1740a1590 0x7fa1740a3a50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa184009380 tx=0x7fa18400b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:09.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.509+0000 7fa17e7fc700 1 -- 172.21.15.27:0/1467195844 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa1800979b0 con 0x7fa188107320 2023-12-17T12:10:09.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:09 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-a[36848]: 2023-12-17T12:10:09.260+0000 7f00edee9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-17T12:10:09.611 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-c[78620]: ignoring --setuser ceph since I am not root 2023-12-17T12:10:09.611 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-c[78620]: ignoring --setgroup ceph since I am not root 2023-12-17T12:10:09.611 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mds[78643]: -- 172.21.15.192:0/3660002025 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f5c2ed5520 con 0x55f5c2eacc00 2023-12-17T12:10:09.611 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-c[78620]: starting mds.c at 2023-12-17T12:10:09.611 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-i[78962]: ignoring --setuser ceph since I am not root 2023-12-17T12:10:09.612 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-i[78962]: ignoring --setgroup ceph since I am not root 2023-12-17T12:10:09.612 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mds[78985]: -- 172.21.15.192:0/1316768358 <== mon.2 v2:172.21.15.141:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f85a37b520 con 0x55f85a352c00 2023-12-17T12:10:09.612 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-i[78962]: starting mds.i at 2023-12-17T12:10:09.612 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-i[78962]: 2023-12-17T12:10:09.329+0000 7f05717e7700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-12-17T12:10:09.612 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-f[78282]: ignoring --setuser ceph since I am not root 2023-12-17T12:10:09.612 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-f[78282]: ignoring --setgroup ceph since I am not root 2023-12-17T12:10:09.612 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mds[78305]: -- 172.21.15.192:0/4231175150 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fe867f7520 con 0x55fe867ce800 2023-12-17T12:10:09.613 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-f[78282]: starting mds.f at 2023-12-17T12:10:09.613 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-e[76755]: ignoring --setuser ceph since I am not root 2023-12-17T12:10:09.613 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-e[76755]: ignoring --setgroup ceph since I am not root 2023-12-17T12:10:09.613 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mds[76778]: -- 172.21.15.141:0/2378192392 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e5229d520 con 0x560e52274800 2023-12-17T12:10:09.613 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-e[76755]: starting mds.e at 2023-12-17T12:10:09.614 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-h[77095]: ignoring --setuser ceph since I am not root 2023-12-17T12:10:09.614 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-h[77095]: ignoring --setgroup ceph since I am not root 2023-12-17T12:10:09.614 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mds[77118]: -- 172.21.15.141:0/3254264108 <== mon.2 v2:172.21.15.141:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561f7217520 con 0x5561f71ef000 2023-12-17T12:10:09.614 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-h[77095]: starting mds.h at 2023-12-17T12:10:09.614 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-b[77436]: ignoring --setuser ceph since I am not root 2023-12-17T12:10:09.614 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-b[77436]: ignoring --setgroup ceph since I am not root 2023-12-17T12:10:09.615 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mds[77459]: -- 172.21.15.141:0/4057415706 <== mon.2 v2:172.21.15.141:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563a8edf9520 con 0x563a8edd0c00 2023-12-17T12:10:09.615 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-b[77436]: starting mds.b at 2023-12-17T12:10:09.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:09.768+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa18810cf00 con 0x7fa188107320 2023-12-17T12:10:09.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:09 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2285805041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:10:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:09 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2658909450' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-17T12:10:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:09 smithi027 ceph-mon[36871]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-17T12:10:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:09 smithi027 ceph-mon[36871]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-17T12:10:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:09 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/2658909450' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-17T12:10:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:09 smithi027 ceph-mon[36871]: osdmap e85: 12 total, 12 up, 12 in 2023-12-17T12:10:09.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:09 smithi027 ceph-mon[36871]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-17T12:10:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2285805041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:10:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2658909450' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-17T12:10:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mon[41225]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-17T12:10:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mon[41225]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-17T12:10:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/2658909450' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-17T12:10:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mon[41225]: osdmap e85: 12 total, 12 up, 12 in 2023-12-17T12:10:10.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:09 smithi141 ceph-mon[41225]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-17T12:10:10.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2285805041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-17T12:10:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2658909450' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-17T12:10:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mon[41899]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-17T12:10:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mon[41899]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-17T12:10:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/2658909450' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-17T12:10:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mon[41899]: osdmap e85: 12 total, 12 up, 12 in 2023-12-17T12:10:10.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:09 smithi192 ceph-mon[41899]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-17T12:10:10.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.264+0000 7fa17e7fc700 1 -- 172.21.15.27:0/1467195844 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa18009afd0 con 0x7fa188107320 2023-12-17T12:10:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa1740a1590 msgr2=0x7fa1740a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa1740a1590 0x7fa1740a3a50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa184009380 tx=0x7fa18400b040 comp rx=0 tx=0).stop 2023-12-17T12:10:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 msgr2=0x7fa1881005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 0x7fa1881005b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa180005910 tx=0x7fa1800057f0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 shutdown_connections 2023-12-17T12:10:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7fa1740a1590 0x7fa1740a3a50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1881020c0 0x7fa1881019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa188107320 0x7fa1881005b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 --2- 172.21.15.27:0/1467195844 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fa1881049f0 0x7fa188100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 >> 172.21.15.27:0/1467195844 conn(0x7fa1880fb9f0 msgr2=0x7fa188102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 shutdown_connections 2023-12-17T12:10:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.269+0000 7fa18f7ff700 1 -- 172.21.15.27:0/1467195844 wait complete. 2023-12-17T12:10:10.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-17T12:10:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 -- 172.21.15.27:0/1497445753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b41015a0 msgr2=0x7f22b41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:10.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 --2- 172.21.15.27:0/1497445753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b41015a0 0x7f22b41019c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22a80099c0 tx=0x7f22a8009cd0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 -- 172.21.15.27:0/1497445753 shutdown_connections 2023-12-17T12:10:10.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 --2- 172.21.15.27:0/1497445753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b4103130 0x7f22b41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 --2- 172.21.15.27:0/1497445753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b41015a0 0x7f22b41019c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 --2- 172.21.15.27:0/1497445753 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f22b4102790 0x7f22b4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 -- 172.21.15.27:0/1497445753 >> 172.21.15.27:0/1497445753 conn(0x7f22b40fcb10 msgr2=0x7f22b40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:10.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 -- 172.21.15.27:0/1497445753 shutdown_connections 2023-12-17T12:10:10.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.508+0000 7f22bafff700 1 -- 172.21.15.27:0/1497445753 wait complete. 2023-12-17T12:10:10.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.509+0000 7f22bafff700 1 Processor -- start 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22bafff700 1 -- start start 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22bafff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b41015a0 0x7f22b4075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22bafff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b4102790 0x7f22b40739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22bafff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f22b4103130 0x7f22b4073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22bafff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40746a0 con 0x7f22b4102790 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22bafff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4074810 con 0x7f22b4103130 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22bafff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b41a14e0 con 0x7f22b41015a0 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b4102790 0x7f22b40739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b4102790 0x7f22b40739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43072/0 (socket says 172.21.15.27:43072) 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.510+0000 7f22b3fff700 1 -- 172.21.15.27:0/807264647 learned_addr learned my addr 172.21.15.27:0/807264647 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b8d9b700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b41015a0 0x7f22b4075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b959c700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f22b4103130 0x7f22b4073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b3fff700 1 -- 172.21.15.27:0/807264647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b41015a0 msgr2=0x7f22b4075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b3fff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b41015a0 0x7f22b4075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b3fff700 1 -- 172.21.15.27:0/807264647 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f22b4103130 msgr2=0x7f22b4073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b3fff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f22b4103130 0x7f22b4073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b3fff700 1 -- 172.21.15.27:0/807264647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a80095f0 con 0x7f22b4102790 2023-12-17T12:10:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b8d9b700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b41015a0 0x7f22b4075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:10:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b3fff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b4102790 0x7f22b40739c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d950 tx=0x7f22a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b1ffb700 1 -- 172.21.15.27:0/807264647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00098e0 con 0x7f22b4102790 2023-12-17T12:10:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b1ffb700 1 -- 172.21.15.27:0/807264647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22a0010460 con 0x7f22b4102790 2023-12-17T12:10:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b41a1770 con 0x7f22b4102790 2023-12-17T12:10:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.511+0000 7f22b1ffb700 1 -- 172.21.15.27:0/807264647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000f620 con 0x7f22b4102790 2023-12-17T12:10:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.512+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b41a1cd0 con 0x7f22b4102790 2023-12-17T12:10:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.512+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b4108ac0 con 0x7f22b4102790 2023-12-17T12:10:10.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.514+0000 7f22b1ffb700 1 -- 172.21.15.27:0/807264647 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f22a00105d0 con 0x7f22b4102790 2023-12-17T12:10:10.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.516+0000 7f22b1ffb700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f22a40a1590 0x7f22a40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:10.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.516+0000 7f22b1ffb700 1 -- 172.21.15.27:0/807264647 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f22a00ccd20 con 0x7f22b4102790 2023-12-17T12:10:10.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.516+0000 7f22b8d9b700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f22a40a1590 0x7f22a40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:10.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.517+0000 7f22b8d9b700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f22a40a1590 0x7f22a40a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22a8000c00 tx=0x7f22a8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:10.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.520+0000 7f22b1ffb700 1 -- 172.21.15.27:0/807264647 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f22a00960d0 con 0x7f22b4102790 2023-12-17T12:10:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:10.737+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f22b4068310 con 0x7f22b4102790 2023-12-17T12:10:10.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1467195844' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: Cluster is now healthy 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/1467195844' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.192:6834/836837634,v1:172.21.15.192:6836/836837634] up:boot 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.192:6835/1869708791,v1:172.21.15.192:6837/1869708791] up:boot 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.192:6838/574097954,v1:172.21.15.192:6839/574097954] up:boot 2023-12-17T12:10:10.912 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6834/3216217859,v1:172.21.15.141:6835/3216217859] up:boot 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6836/2716259584,v1:172.21.15.141:6837/2716259584] up:boot 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: mds.? [v2:172.21.15.141:6838/4271335388,v1:172.21.15.141:6839/4271335388] up:boot 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: fsmap 12 up:standby 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-17T12:10:10.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:10 smithi027 ceph-mon[36871]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-17T12:10:11.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:11.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1467195844' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:10:11.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-17T12:10:11.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-17T12:10:11.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: Cluster is now healthy 2023-12-17T12:10:11.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/1467195844' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-17T12:10:11.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.192:6834/836837634,v1:172.21.15.192:6836/836837634] up:boot 2023-12-17T12:10:11.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.192:6835/1869708791,v1:172.21.15.192:6837/1869708791] up:boot 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.192:6838/574097954,v1:172.21.15.192:6839/574097954] up:boot 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6834/3216217859,v1:172.21.15.141:6835/3216217859] up:boot 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6836/2716259584,v1:172.21.15.141:6837/2716259584] up:boot 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: mds.? [v2:172.21.15.141:6838/4271335388,v1:172.21.15.141:6839/4271335388] up:boot 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: fsmap 12 up:standby 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-17T12:10:11.015 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-17T12:10:11.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-17T12:10:11.016 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:10 smithi141 ceph-mon[41225]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-17T12:10:11.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-17T12:10:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1467195844' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-17T12:10:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-17T12:10:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-17T12:10:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: Cluster is now healthy 2023-12-17T12:10:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/1467195844' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-17T12:10:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.192:6834/836837634,v1:172.21.15.192:6836/836837634] up:boot 2023-12-17T12:10:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.192:6835/1869708791,v1:172.21.15.192:6837/1869708791] up:boot 2023-12-17T12:10:11.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.192:6838/574097954,v1:172.21.15.192:6839/574097954] up:boot 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6834/3216217859,v1:172.21.15.141:6835/3216217859] up:boot 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6836/2716259584,v1:172.21.15.141:6837/2716259584] up:boot 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: mds.? [v2:172.21.15.141:6838/4271335388,v1:172.21.15.141:6839/4271335388] up:boot 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: fsmap 12 up:standby 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-17T12:10:11.062 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:10 smithi192 ceph-mon[41899]: from='mgr.14154 172.21.15.27:0/2866977541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-17T12:10:11.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.620+0000 7f22b1ffb700 1 -- 172.21.15.27:0/807264647 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f22a00160d0 con 0x7f22b4102790 2023-12-17T12:10:11.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.624+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f22a40a1590 msgr2=0x7f22a40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:11.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.624+0000 7f22bafff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f22a40a1590 0x7f22a40a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22a8000c00 tx=0x7f22a8005fb0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.624+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b4102790 msgr2=0x7f22b40739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:11.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.624+0000 7f22bafff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b4102790 0x7f22b40739c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d950 tx=0x7f22a000dc60 comp rx=0 tx=0).stop 2023-12-17T12:10:11.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.625+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 shutdown_connections 2023-12-17T12:10:11.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.625+0000 7f22bafff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7f22a40a1590 0x7f22a40a3a50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.625+0000 7f22bafff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b41015a0 0x7f22b4075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.625+0000 7f22bafff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f22b4103130 0x7f22b4073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.625+0000 7f22bafff700 1 --2- 172.21.15.27:0/807264647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b4102790 0x7f22b40739c0 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d950 tx=0x7f22a000dc60 comp rx=0 tx=0).stop 2023-12-17T12:10:11.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.625+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 >> 172.21.15.27:0/807264647 conn(0x7f22b40fcb10 msgr2=0x7f22b4105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:11.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.625+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 shutdown_connections 2023-12-17T12:10:11.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.625+0000 7f22bafff700 1 -- 172.21.15.27:0/807264647 wait complete. 2023-12-17T12:10:11.629 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-17T12:10:11.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-17T12:10:11.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 -- 172.21.15.27:0/3158372804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8107260 msgr2=0x7efdd81096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 --2- 172.21.15.27:0/3158372804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8107260 0x7efdd81096d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efdd000b0a0 tx=0x7efdd000b3b0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 -- 172.21.15.27:0/3158372804 shutdown_connections 2023-12-17T12:10:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 --2- 172.21.15.27:0/3158372804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8107260 0x7efdd81096d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 --2- 172.21.15.27:0/3158372804 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdd8102000 0x7efdd81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 --2- 172.21.15.27:0/3158372804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd8104930 0x7efdd8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 -- 172.21.15.27:0/3158372804 >> 172.21.15.27:0/3158372804 conn(0x7efdd80fb9f0 msgr2=0x7efdd80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 -- 172.21.15.27:0/3158372804 shutdown_connections 2023-12-17T12:10:11.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.869+0000 7efddcfff700 1 -- 172.21.15.27:0/3158372804 wait complete. 2023-12-17T12:10:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.870+0000 7efddcfff700 1 Processor -- start 2023-12-17T12:10:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.870+0000 7efddcfff700 1 -- start start 2023-12-17T12:10:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efddcfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102000 0x7efdd81974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efddcfff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdd8104930 0x7efdd8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efddcfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd8107260 0x7efdd819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efddcfff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd819f000 con 0x7efdd8102000 2023-12-17T12:10:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efddcfff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd819f140 con 0x7efdd8104930 2023-12-17T12:10:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efddcfff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd819f2d0 con 0x7efdd8107260 2023-12-17T12:10:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102000 0x7efdd81974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102000 0x7efdd81974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43082/0 (socket says 172.21.15.27:43082) 2023-12-17T12:10:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd659c700 1 -- 172.21.15.27:0/4008905267 learned_addr learned my addr 172.21.15.27:0/4008905267 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-17T12:10:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd6d9d700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd8107260 0x7efdd819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd5d9b700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdd8104930 0x7efdd8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd659c700 1 -- 172.21.15.27:0/4008905267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd8107260 msgr2=0x7efdd819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd659c700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd8107260 0x7efdd819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd659c700 1 -- 172.21.15.27:0/4008905267 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdd8104930 msgr2=0x7efdd8197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd659c700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdd8104930 0x7efdd8197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd659c700 1 -- 172.21.15.27:0/4008905267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0009d00 con 0x7efdd8102000 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.871+0000 7efdd6d9d700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd8107260 0x7efdd819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.872+0000 7efdd659c700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102000 0x7efdd81974d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efdc800e9a0 tx=0x7efdc800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.872+0000 7efdc77fe700 1 -- 172.21.15.27:0/4008905267 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8009790 con 0x7efdd8102000 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.872+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd819f5b0 con 0x7efdd8102000 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.872+0000 7efdc77fe700 1 -- 172.21.15.27:0/4008905267 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdc800cbd0 con 0x7efdd8102000 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.872+0000 7efdc77fe700 1 -- 172.21.15.27:0/4008905267 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8018680 con 0x7efdd8102000 2023-12-17T12:10:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.872+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd819fac0 con 0x7efdd8102000 2023-12-17T12:10:11.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.873+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd80fd570 con 0x7efdd8102000 2023-12-17T12:10:11.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.875+0000 7efdc77fe700 1 -- 172.21.15.27:0/4008905267 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7efdc800cd40 con 0x7efdd8102000 2023-12-17T12:10:11.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.878+0000 7efdc77fe700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7efdc00a1510 0x7efdc00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-17T12:10:11.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.878+0000 7efdc77fe700 1 -- 172.21.15.27:0/4008905267 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7886+0+0 (secure 0 0 0) 0x7efdc8014070 con 0x7efdd8102000 2023-12-17T12:10:11.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.878+0000 7efdd5d9b700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7efdc00a1510 0x7efdc00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-17T12:10:11.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.878+0000 7efdd5d9b700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7efdc00a1510 0x7efdc00a39d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009bf0 tx=0x7efdcc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-17T12:10:11.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:11.882+0000 7efdc77fe700 1 -- 172.21.15.27:0/4008905267 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efdc80522f0 con 0x7efdd8102000 2023-12-17T12:10:11.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:11 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/807264647' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-17T12:10:12.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:11 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/807264647' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-17T12:10:12.060 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:11 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/807264647' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-17T12:10:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.093+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efdd810ce60 con 0x7efdd8102000 2023-12-17T12:10:12.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.623+0000 7efdc77fe700 1 -- 172.21.15.27:0/4008905267 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7efdc8096690 con 0x7efdd8102000 2023-12-17T12:10:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7efdc00a1510 msgr2=0x7efdc00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7efdc00a1510 0x7efdc00a39d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009bf0 tx=0x7efdcc009260 comp rx=0 tx=0).stop 2023-12-17T12:10:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102000 msgr2=0x7efdd81974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-17T12:10:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102000 0x7efdd81974d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efdc800e9a0 tx=0x7efdc800ed60 comp rx=0 tx=0).stop 2023-12-17T12:10:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 shutdown_connections 2023-12-17T12:10:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:6800/46170456,v1:172.21.15.27:6801/46170456] conn(0x7efdc00a1510 0x7efdc00a39d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd8107260 0x7efdd819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd8102000 0x7efdd81974d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 --2- 172.21.15.27:0/4008905267 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdd8104930 0x7efdd8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-17T12:10:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.627+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 >> 172.21.15.27:0/4008905267 conn(0x7efdd80fb9f0 msgr2=0x7efdd8102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-17T12:10:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.628+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 shutdown_connections 2023-12-17T12:10:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-17T12:10:12.628+0000 7efddcfff700 1 -- 172.21.15.27:0/4008905267 wait complete. 2023-12-17T12:10:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-17T12:10:12.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:10:12.726 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:10:12.811 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:10:12.884 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:12 smithi192 ceph-mon[41899]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-12-17T12:10:12.884 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:12 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/807264647' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-17T12:10:12.885 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:12 smithi192 ceph-mon[41899]: osdmap e86: 12 total, 12 up, 12 in 2023-12-17T12:10:12.885 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:12 smithi192 ceph-mon[41899]: from='client.? 172.21.15.27:0/4008905267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-17T12:10:12.891 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-17T12:10:12.892 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-17T12:10:12.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:12 smithi027 ceph-mon[36871]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-12-17T12:10:12.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:12 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/807264647' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-17T12:10:12.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:12 smithi027 ceph-mon[36871]: osdmap e86: 12 total, 12 up, 12 in 2023-12-17T12:10:12.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:12 smithi027 ceph-mon[36871]: from='client.? 172.21.15.27:0/4008905267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-17T12:10:12.936 DEBUG:teuthology.orchestra.run.smithi141:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-17T12:10:12.982 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-17T12:10:13.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:12 smithi141 ceph-mon[41225]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-12-17T12:10:13.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:12 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/807264647' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-17T12:10:13.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:12 smithi141 ceph-mon[41225]: osdmap e86: 12 total, 12 up, 12 in 2023-12-17T12:10:13.014 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:12 smithi141 ceph-mon[41225]: from='client.? 172.21.15.27:0/4008905267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-17T12:10:13.026 INFO:tasks.cephadm:Stopping all daemons... 2023-12-17T12:10:13.027 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-17T12:10:13.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.a 2023-12-17T12:10:13.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:13 smithi027 systemd[1]: Stopping Ceph mon.a for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:13.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-a[36848]: 2023-12-17T12:10:13.448+0000 7f00f36f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:13.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-a[36848]: 2023-12-17T12:10:13.448+0000 7f00f36f4700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-17T12:10:14.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Dec 17 12:10:13 smithi027 bash[100269]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-a 2023-12-17T12:10:14.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.a.service' 2023-12-17T12:10:14.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:14.351 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-17T12:10:14.351 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-12-17T12:10:14.352 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.b 2023-12-17T12:10:14.764 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:14 smithi141 systemd[1]: Stopping Ceph mon.b for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:15.111 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-b[41202]: 2023-12-17T12:10:14.852+0000 7feb07d72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:15.111 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-b[41202]: 2023-12-17T12:10:14.852+0000 7feb07d72700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-12-17T12:10:15.513 INFO:journalctl@ceph.mon.b.smithi141.stdout:Dec 17 12:10:15 smithi141 bash[79140]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-b 2023-12-17T12:10:15.622 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.b.service' 2023-12-17T12:10:15.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:15.741 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-12-17T12:10:15.741 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-12-17T12:10:15.742 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.c 2023-12-17T12:10:16.061 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:15 smithi192 systemd[1]: Stopping Ceph mon.c for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:16.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:16 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-c[41869]: 2023-12-17T12:10:16.150+0000 7f08ce4d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:16.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:16 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-c[41869]: 2023-12-17T12:10:16.150+0000 7f08ce4d5700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-12-17T12:10:16.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Dec 17 12:10:16 smithi192 bash[80894]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mon-c 2023-12-17T12:10:16.813 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mon.c.service' 2023-12-17T12:10:16.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:16.913 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-12-17T12:10:16.913 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-17T12:10:16.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.x 2023-12-17T12:10:17.315 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:10:17 smithi027 systemd[1]: Stopping Ceph mgr.x for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:17.579 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:10:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:10:17.314+0000 7fcf12e1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:17.579 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:10:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x[37122]: 2023-12-17T12:10:17.314+0000 7fcf12e1b700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-17T12:10:17.911 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Dec 17 12:10:17 smithi027 bash[100763]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-x 2023-12-17T12:10:17.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.x.service' 2023-12-17T12:10:18.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:18.058 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-17T12:10:18.058 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-17T12:10:18.059 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.y 2023-12-17T12:10:18.514 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:10:18 smithi141 systemd[1]: Stopping Ceph mgr.y for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:19.013 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:10:18 smithi141 bash[79412]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-y 2023-12-17T12:10:19.014 INFO:journalctl@ceph.mgr.y.smithi141.stdout:Dec 17 12:10:18 smithi141 systemd[1]: ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-12-17T12:10:19.114 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.y.service' 2023-12-17T12:10:19.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:19.200 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-17T12:10:19.200 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-12-17T12:10:19.200 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.z 2023-12-17T12:10:19.560 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:10:19 smithi192 systemd[1]: Stopping Ceph mgr.z for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:20.060 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:10:19 smithi192 bash[81043]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mgr-z 2023-12-17T12:10:20.060 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Dec 17 12:10:20 smithi192 systemd[1]: ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-12-17T12:10:20.298 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mgr.z.service' 2023-12-17T12:10:20.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:20.385 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-12-17T12:10:20.385 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-17T12:10:20.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.0 2023-12-17T12:10:20.772 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:10:20 smithi027 systemd[1]: Stopping Ceph osd.0 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:21.161 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:10:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-0[55535]: 2023-12-17T12:10:20.772+0000 7f6cc706c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:21.161 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:10:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-0[55535]: 2023-12-17T12:10:20.772+0000 7f6cc706c700 -1 osd.0 87 *** Got signal Terminated *** 2023-12-17T12:10:21.162 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:10:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-0[55535]: 2023-12-17T12:10:20.772+0000 7f6cc706c700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:10:26.661 INFO:journalctl@ceph.osd.0.smithi027.stdout:Dec 17 12:10:26 smithi027 bash[100906]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-0 2023-12-17T12:10:27.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.0.service' 2023-12-17T12:10:28.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:28.092 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-17T12:10:28.092 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-17T12:10:28.093 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.1 2023-12-17T12:10:28.475 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:10:28 smithi141 systemd[1]: Stopping Ceph osd.1 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:28.764 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:10:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-1[46239]: 2023-12-17T12:10:28.474+0000 7f7e556fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:28.764 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:10:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-1[46239]: 2023-12-17T12:10:28.474+0000 7f7e556fc700 -1 osd.1 87 *** Got signal Terminated *** 2023-12-17T12:10:28.764 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:10:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-1[46239]: 2023-12-17T12:10:28.474+0000 7f7e556fc700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:10:34.514 INFO:journalctl@ceph.osd.1.smithi141.stdout:Dec 17 12:10:34 smithi141 bash[79558]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-1 2023-12-17T12:10:35.760 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.1.service' 2023-12-17T12:10:35.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:35.850 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-17T12:10:35.850 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-17T12:10:35.850 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.2 2023-12-17T12:10:36.223 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:10:35 smithi192 systemd[1]: Stopping Ceph osd.2 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:36.560 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:10:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-2[47711]: 2023-12-17T12:10:36.222+0000 7fdfccd39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:36.560 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:10:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-2[47711]: 2023-12-17T12:10:36.222+0000 7fdfccd39700 -1 osd.2 87 *** Got signal Terminated *** 2023-12-17T12:10:36.560 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:10:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-2[47711]: 2023-12-17T12:10:36.222+0000 7fdfccd39700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:10:42.310 INFO:journalctl@ceph.osd.2.smithi192.stdout:Dec 17 12:10:41 smithi192 bash[81185]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-2 2023-12-17T12:10:43.559 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.2.service' 2023-12-17T12:10:43.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:43.650 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-17T12:10:43.650 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-17T12:10:43.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.3 2023-12-17T12:10:44.036 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:10:43 smithi027 systemd[1]: Stopping Ceph osd.3 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:44.411 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:10:44 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3[63997]: 2023-12-17T12:10:44.034+0000 7f5422a7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:44.411 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:10:44 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3[63997]: 2023-12-17T12:10:44.034+0000 7f5422a7e700 -1 osd.3 87 *** Got signal Terminated *** 2023-12-17T12:10:44.412 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:10:44 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3[63997]: 2023-12-17T12:10:44.034+0000 7f5422a7e700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:10:47.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:46.634+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:47.411 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:46 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:46.982+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:48.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:47 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:47.647+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:48.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:47 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:47.948+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:48.162 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:10:47 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3[63997]: 2023-12-17T12:10:47.827+0000 7f541d075700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.255475+0000 front 2023-12-17T12:10:22.255518+0000 (oldest deadline 2023-12-17T12:10:46.955629+0000) 2023-12-17T12:10:48.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:47 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:47.912+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:48.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:48.666+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:48.811 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:48.502+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:49.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:48.969+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:49.162 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:10:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3[63997]: 2023-12-17T12:10:48.806+0000 7f541d075700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.255475+0000 front 2023-12-17T12:10:22.255518+0000 (oldest deadline 2023-12-17T12:10:46.955629+0000) 2023-12-17T12:10:49.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:48 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:48.927+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:49.810 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:49.552+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:49.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:49.710+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:49.880 INFO:journalctl@ceph.osd.3.smithi027.stdout:Dec 17 12:10:49 smithi027 bash[101169]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-3 2023-12-17T12:10:50.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:49 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:49.933+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:50.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:49 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:49.877+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:51.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:50 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:50.716+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:51.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:50 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:50.578+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:51.089 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:50 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:50.973+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:51.089 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:50 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:50.869+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:51.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.3.service' 2023-12-17T12:10:51.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:51.209 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-17T12:10:51.209 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-17T12:10:51.209 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.4 2023-12-17T12:10:51.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:51 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:51.169+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:51.514 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:10:51 smithi141 systemd[1]: Stopping Ceph osd.4 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:52.013 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:10:51 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-4[54156]: 2023-12-17T12:10:51.604+0000 7f7ae573d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:52.014 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:10:51 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-4[54156]: 2023-12-17T12:10:51.604+0000 7f7ae573d700 -1 osd.4 87 *** Got signal Terminated *** 2023-12-17T12:10:52.014 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:10:51 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-4[54156]: 2023-12-17T12:10:51.604+0000 7f7ae573d700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:10:52.060 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:51 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:51.611+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:52.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:51 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:51.740+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:52.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:51 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:51.996+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:52.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:51 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:51.867+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:52.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:52 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:52.158+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:53.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:52 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:52.788+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:53.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:52 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:52.636+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:53.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:52 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:52.996+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:53.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:52 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:52.910+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:53.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:53 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:53.149+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:54.060 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:53 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:53.657+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:54.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:53 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:53.746+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:54.411 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:53 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:53.995+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:54.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:53 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:53.933+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:54.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:54 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:54.100+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:55.060 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:54 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:54.618+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:55.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:54 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:54.713+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:55.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:10:54 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:10:54.775+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:10:55.412 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:54 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:54.964+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:55.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:54 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:54.957+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:55.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:55 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:55.093+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:56.013 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:10:55 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:10:55.561+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:10:56.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:10:55 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:10:55.749+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:10:56.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:55 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:55.710+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:56.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:55 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:55.710+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:10:56.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:55 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:55.639+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:56.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:55 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:55.639+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:10:56.411 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:55 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:55.985+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:56.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:55 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:55.988+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:56.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:56 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:56.096+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:57.014 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:10:56 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:10:56.545+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:10:57.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:10:56 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:10:56.760+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:10:57.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:56 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:56.699+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:57.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:56 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:56.699+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:10:57.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:56 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:56.592+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:57.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:56 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:56.592+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:10:57.411 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:56 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:56.946+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:57.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:57 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:57.020+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:57.513 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:10:57 smithi141 bash[79822]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-4 2023-12-17T12:10:57.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:57 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:57.131+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:57.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:57 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:57.131+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:10:57.838 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:10:57 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:10:57.563+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:10:58.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:10:57 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:10:57.733+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:10:58.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:57 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:57.701+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:58.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:57 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:57.701+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:10:58.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:57 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:57.582+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:58.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:57 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:57.582+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:10:58.411 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:57 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:57.916+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:58.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:57 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:57.983+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:58.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:57 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:57.983+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:10:58.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:58 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:58.154+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:58.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:58 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:58.154+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:10:58.699 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.4.service' 2023-12-17T12:10:58.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:10:58 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:10:58.537+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:10:58.764 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:10:58 smithi141 systemd[1]: ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.4.service: Succeeded. 2023-12-17T12:10:58.764 INFO:journalctl@ceph.osd.4.smithi141.stdout:Dec 17 12:10:58 smithi141 systemd[1]: Stopped Ceph osd.4 for 62030760-9cd3-11ee-95a5-87774f69a715. 2023-12-17T12:10:58.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:10:58.789 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-17T12:10:58.789 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-17T12:10:58.789 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.5 2023-12-17T12:10:58.810 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:10:58 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:10:58.717+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:10:58.810 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:58 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:58.541+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:58.810 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:58 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:58.541+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:10:58.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:58 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:58.723+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:58.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:58 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:58.723+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:10:59.169 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:58 smithi192 systemd[1]: Stopping Ceph osd.5 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:10:59.411 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:58 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:58.912+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:10:59.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:59.022+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:10:59.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:59.022+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:10:59.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:59 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:59.152+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:10:59.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:10:59 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:10:59.152+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:10:59.521 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:59.168+0000 7f8c0fff4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:10:59.521 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:59.168+0000 7f8c0fff4700 -1 osd.5 87 *** Got signal Terminated *** 2023-12-17T12:10:59.521 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:59.168+0000 7f8c0fff4700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:10:59.521 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:59.520+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:10:59.810 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:10:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:10:59.520+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:10:59.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:10:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:10:59.682+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:10:59.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:59.734+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:10:59.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:10:59 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:10:59.734+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:00.013 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:10:59 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:10:59.582+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:00.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:59.879+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:00.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:10:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:10:59.879+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:00.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:59.997+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:00.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:10:59 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:10:59.997+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:00.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:00 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:00.143+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:00.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:00 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:00.143+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:00.810 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:00 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:00.644+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:00.811 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:00 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:00.555+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:11:00.811 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:00 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:00.555+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:11:00.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:00 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:00.751+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:00.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:00 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:00.751+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:01.013 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:00 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:00.628+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:01.411 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:00 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:00.922+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:01.412 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:00 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:00.922+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:01.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:01 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:01.042+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:01.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:01 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:01.042+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:01.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:01 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:01.100+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:01.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:01 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:01.100+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:02.014 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:01 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:01.588+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:02.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:01 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:01.619+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:02.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:01 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:01.573+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:11:02.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:01 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:01.573+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:11:02.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:01 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:01.715+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:02.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:01 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:01.715+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:02.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:01 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:01.905+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:02.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:01 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:01.905+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:02.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:02 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:02.051+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:02.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:02 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:02.051+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:02.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:02 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:02.056+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:02.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:02 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:02.056+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:03.013 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:02 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:02.547+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:03.060 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:02 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:02.590+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:11:03.060 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:02 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:02.590+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:11:03.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:02 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:02.739+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:03.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:02 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:02.739+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:03.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:02 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:02.590+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:03.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:02 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:02.590+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:03.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:02 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:02.903+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:03.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:02 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:02.903+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:03.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:02 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:02.903+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:03.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:03.011+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:03.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:03.011+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:03.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:03 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:03.083+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:03.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:03 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:03.083+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:04.013 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:03 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:03.553+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:04.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:03 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:03.618+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:04.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:03 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:03.618+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:04.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:03 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:03.606+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.883776+0000 front 2023-12-17T12:10:20.883794+0000 (oldest deadline 2023-12-17T12:10:48.484531+0000) 2023-12-17T12:11:04.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:03 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:03.606+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.986125+0000 front 2023-12-17T12:10:31.986016+0000 (oldest deadline 2023-12-17T12:10:55.485800+0000) 2023-12-17T12:11:04.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:03 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5[56099]: 2023-12-17T12:11:03.607+0000 7f8c0b5ed700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.786859+0000 front 2023-12-17T12:10:40.786904+0000 (oldest deadline 2023-12-17T12:11:03.086948+0000) 2023-12-17T12:11:04.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:03 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:03.740+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:04.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:03 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:03.740+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:04.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:03 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:03.740+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:04.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:03.895+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:04.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:03.895+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:04.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:03.895+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:04.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:03.989+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:04.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:03.989+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:04.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:03.989+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:04.509 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:04 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:04.094+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:04.510 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:04 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:04.094+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:04.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:04 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:04.509+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:05.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:04 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:04.618+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:05.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:04 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:04.618+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:05.061 INFO:journalctl@ceph.osd.5.smithi192.stdout:Dec 17 12:11:04 smithi192 bash[81449]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-5 2023-12-17T12:11:05.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:04 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:04.720+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:05.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:04 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:04.720+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:05.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:04 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:04.720+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:05.411 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:04 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:04.923+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:05.412 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:04 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:04.923+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:05.412 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:04 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:04.923+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:05.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:04 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:04.975+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:05.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:04 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:04.975+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:05.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:04 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:04.975+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:05.479 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:05 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:05.061+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:05.479 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:05 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:05.061+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:05.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:05 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:05.479+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:05.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:05 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:05.479+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:05.810 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:05 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:05.622+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:05.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:05 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:05.622+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:05.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:05 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:05.679+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:05.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:05 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:05.679+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:05.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:05 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:05.679+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:06.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:05 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:05.877+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:06.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:05 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:05.877+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:06.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:05 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:05.877+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:06.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:05 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:05.932+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:06.163 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:05 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:05.932+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:06.163 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:05 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:05.932+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:06.293 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.5.service' 2023-12-17T12:11:06.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:11:06.422 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-17T12:11:06.422 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-17T12:11:06.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.6 2023-12-17T12:11:06.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:06 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:06.042+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:06.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:06 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:06.042+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:06.808 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:06 smithi027 systemd[1]: Stopping Ceph osd.6 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:11:07.010 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:06 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:06.517+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:07.010 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:06 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:06.517+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:07.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:06 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:06.621+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:07.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:06 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:06.621+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:07.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:06 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:06.713+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:07.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:06 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:06.713+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:07.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:06 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:06.713+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:07.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:06.939+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:07.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:06.939+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:07.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:06.939+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:07.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:06.807+0000 7f83c8e67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:11:07.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:06.807+0000 7f83c8e67700 -1 osd.6 87 *** Got signal Terminated *** 2023-12-17T12:11:07.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:06.807+0000 7f83c8e67700 -1 osd.6 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:11:07.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:06.891+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:07.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:06.891+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:07.163 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:06 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:06.891+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:07.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:07 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:07.009+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:07.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:07 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:07.009+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:07.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:07 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:07.009+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:07.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:07 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:07.474+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:07.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:07 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:07.474+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:08.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:07 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:07.594+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:08.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:07 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:07.594+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:08.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:07 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:07.721+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:08.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:07 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:07.721+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:08.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:07 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:07.721+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:08.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:07.853+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:08.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:07.853+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:08.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:07.853+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:08.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:07.953+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:08.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:07.953+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:08.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:07.953+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:08.509 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:08 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:08.020+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:08.509 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:08 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:08.020+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:08.510 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:08 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:08.020+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:08.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:08 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:08.509+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:08.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:08 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:08.509+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:09.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:08 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:08.614+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:09.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:08 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:08.614+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:09.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:08 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:08.672+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:09.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:08 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:08.672+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:09.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:08 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:08.672+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:09.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:08 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:08.882+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:09.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:08 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:08.882+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:09.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:08 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:08.882+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:09.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:08 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:08.909+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:09.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:08 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:08.909+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:09.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:08 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:08.909+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:09.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:08 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:08.995+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:09.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:08 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:08.995+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:09.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:08 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:08.995+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:10.011 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:09.526+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:10.012 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:09 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:09.526+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:10.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:09.623+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:10.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:09.623+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:10.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:09.681+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:10.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:09.681+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:10.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:09 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:09.681+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:10.161 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:09 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:09.850+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:10.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:09 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:09.850+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:10.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:09 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:09.850+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:10.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:09 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:09.939+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:10.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:09 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:09.939+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:10.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:09 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:09.939+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:10.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:10.010+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:10.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:10.010+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:10.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:10.010+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:10.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:10.504+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:10.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:10.504+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:10.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:10.504+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:11.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:10 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:10.669+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:11.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:10 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:10.669+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:11.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:10 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:10.669+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:11.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:10 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:10.669+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:11.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:10 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:10.669+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:11.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:10 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:10.840+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:21.176449+0000 front 2023-12-17T12:10:21.176484+0000 (oldest deadline 2023-12-17T12:10:46.977382+0000) 2023-12-17T12:11:11.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:10 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:10.840+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:29.278853+0000 front 2023-12-17T12:10:29.278732+0000 (oldest deadline 2023-12-17T12:10:59.279873+0000) 2023-12-17T12:11:11.162 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:10 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6[72510]: 2023-12-17T12:11:10.840+0000 7f83c145a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.381479+0000 front 2023-12-17T12:10:40.381309+0000 (oldest deadline 2023-12-17T12:11:02.581737+0000) 2023-12-17T12:11:11.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:10 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:10.928+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:11.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:10 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:10.928+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:11.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:10 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:10.928+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:11.163 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:10 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:10.928+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:11.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:10.986+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:11.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:10.986+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:11.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:10 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:10.986+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:11.975 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:11 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:11.529+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:11.975 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:11 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:11.529+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:11.975 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:11 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:11.529+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:12.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:11 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:11.626+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:12.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:11 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:11.626+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:12.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:11 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:11.678+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:12.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:11 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:11.678+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:12.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:11 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:11.678+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:12.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:11 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:11.900+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:12.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:11 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:11.900+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:12.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:11 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:11.900+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:12.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:11 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:11.900+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:12.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:11 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:11.974+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:12.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:11 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:11.974+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:12.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:11 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:11.974+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:12.661 INFO:journalctl@ceph.osd.6.smithi027.stdout:Dec 17 12:11:12 smithi027 bash[101435]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-6 2023-12-17T12:11:12.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:12 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:12.492+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:12.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:12 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:12.492+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:12.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:12 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:12.492+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:13.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:12 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:12.660+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:13.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:12 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:12.660+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:13.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:12 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:12.728+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:13.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:12 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:12.728+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:13.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:12 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:12.728+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:13.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:12 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:12.903+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:13.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:12 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:12.903+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:13.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:12 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:12.903+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:13.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:12 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:12.903+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:13.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:13 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:13.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:13.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:13 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:13.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:13.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:13 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:13.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:13.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:13 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:13.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:13.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.6.service' 2023-12-17T12:11:14.014 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:13 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:13.534+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:14.014 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:13 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:13.534+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:14.014 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:13 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:13.534+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:14.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:11:14.028 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-17T12:11:14.028 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-17T12:11:14.029 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.7 2023-12-17T12:11:14.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:13 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:13.737+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:14.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:13 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:13.737+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:14.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:13 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:13.737+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:14.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:13 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:13.612+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:14.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:13 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:13.612+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:14.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:13 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:13.612+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:14.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:13.861+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:14.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:13.861+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:14.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:13.861+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:14.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:13 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:13.861+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:14.417 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:14 smithi141 systemd[1]: Stopping Ceph osd.7 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:11:14.417 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:14.022+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:14.418 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:14.022+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:14.418 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:14.022+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:14.418 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:14.022+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:14.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:14.416+0000 7f7aaaca0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:11:14.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:14.416+0000 7f7aaaca0700 -1 osd.7 87 *** Got signal Terminated *** 2023-12-17T12:11:14.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:14.416+0000 7f7aaaca0700 -1 osd.7 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:11:14.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:14.523+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:14.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:14.523+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:14.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:14 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:14.523+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:15.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:14 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:14.726+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:15.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:14 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:14.726+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:15.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:14 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:14.726+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:15.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:14 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:14.567+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:15.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:14 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:14.567+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:15.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:14 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:14.567+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:15.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:14.884+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:15.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:14.884+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:15.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:14.884+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:15.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:14 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:14.884+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:15.501 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:15 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:15.052+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:15.501 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:15 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:15.052+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:15.501 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:15 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:15.052+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:15.501 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:15 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:15.052+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:15.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:15 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:15.500+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:15.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:15 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:15.500+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:15.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:15 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:15.500+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:16.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:15 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:15.722+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:16.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:15 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:15.722+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:16.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:15 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:15.722+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:16.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:15 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:15.593+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:16.062 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:15 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:15.593+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:16.062 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:15 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:15.593+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:16.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:15 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:15.901+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:16.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:15 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:15.901+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:16.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:15 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:15.901+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:16.162 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:15 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:15.901+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:16.476 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:16.017+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:16.476 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:16.017+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:16.476 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:16.017+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:16.476 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:16.017+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:16.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:16.475+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:16.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:16.475+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:16.763 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:16.475+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:17.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:16 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:16.758+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:17.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:16 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:16.758+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:17.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:16 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:16.758+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:17.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:16 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:16.581+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:17.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:16 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:16.581+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:17.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:16 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:16.581+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:17.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:16.977+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:17.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:16.977+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:17.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:16.977+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:17.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:16 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:16.977+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:17.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:16.947+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:17.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:16.948+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:17.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:16.948+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:17.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:16 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:16.948+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:17.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:17.496+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:17.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:17.496+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:17.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:17.496+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:17.764 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:17.496+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.991044+0000 front 2023-12-17T12:10:51.991071+0000 (oldest deadline 2023-12-17T12:11:17.291126+0000) 2023-12-17T12:11:18.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:17 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:17.569+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:18.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:17 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:17.569+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:18.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:17 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:17.569+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:18.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:17 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:17.743+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:18.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:17 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:17.743+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:18.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:17 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:17.743+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:18.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:17 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:17.743+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:18.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:18.008+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:18.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:18.008+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:18.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:18.008+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:18.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:18.008+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:18.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:17.914+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:18.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:17.914+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:18.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:17.914+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:18.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:17 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:17.914+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:18.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:18 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:18.783+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:18.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:18 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:18.783+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:18.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:18 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:18.783+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:18.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:18 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:18.783+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:18.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:18 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:18.549+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:18.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:18 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:18.549+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:18.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:18 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:18.549+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:18.989 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:18.525+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.586115+0000 front 2023-12-17T12:10:30.586140+0000 (oldest deadline 2023-12-17T12:10:54.686281+0000) 2023-12-17T12:11:18.990 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:18.525+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:35.787518+0000 front 2023-12-17T12:10:35.787578+0000 (oldest deadline 2023-12-17T12:11:04.588145+0000) 2023-12-17T12:11:18.990 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:18.525+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:47.389805+0000 front 2023-12-17T12:10:47.389691+0000 (oldest deadline 2023-12-17T12:11:10.289864+0000) 2023-12-17T12:11:18.990 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7[62744]: 2023-12-17T12:11:18.525+0000 7f7aa729b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.991044+0000 front 2023-12-17T12:10:51.991071+0000 (oldest deadline 2023-12-17T12:11:17.291126+0000) 2023-12-17T12:11:19.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:18.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:19.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:18.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:19.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:18.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:19.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:18 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:18.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:19.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:18 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:18.943+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:19.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:18 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:18.943+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:19.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:18 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:18.943+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:19.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:18 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:18.943+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:20.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:19 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:19.748+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:20.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:19 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:19.748+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:20.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:19 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:19.748+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:20.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:19 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:19.748+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:20.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:19 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:19.561+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:20.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:19 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:19.561+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:20.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:19 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:19.561+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:20.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:19 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:19.561+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.728872+0000 front 2023-12-17T12:10:52.728848+0000 (oldest deadline 2023-12-17T12:11:18.628743+0000) 2023-12-17T12:11:20.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:19 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:19.973+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:20.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:19 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:19.973+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:20.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:19 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:19.973+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:20.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:19 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:19.973+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:20.513 INFO:journalctl@ceph.osd.7.smithi141.stdout:Dec 17 12:11:20 smithi141 bash[80076]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-7 2023-12-17T12:11:20.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.025+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:20.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.025+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:20.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.025+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:20.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.025+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:20.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:20 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:20.751+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:20.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:20 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:20.751+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:20.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:20 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:20.751+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:20.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:20 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:20.751+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:20.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:20 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:20.512+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:20.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:20 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:20.512+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:20.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:20 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:20.512+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:20.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:20 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:20.512+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.728872+0000 front 2023-12-17T12:10:52.728848+0000 (oldest deadline 2023-12-17T12:11:18.628743+0000) 2023-12-17T12:11:21.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.984+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:21.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.984+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:21.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.984+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:21.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.984+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:21.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:20 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:20.984+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:21.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:20.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:21.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:20.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:21.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:20.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:21.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:20 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:20.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:21.762 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.7.service' 2023-12-17T12:11:21.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:21 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:21.715+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:21.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:21 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:21.715+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:21.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:21 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:21.715+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:21.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:21 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:21.715+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:21.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:21 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:21.558+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:21.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:21 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:21.558+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:21.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:21 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:21.558+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:21.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:21 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:21.558+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.728872+0000 front 2023-12-17T12:10:52.728848+0000 (oldest deadline 2023-12-17T12:11:18.628743+0000) 2023-12-17T12:11:21.851 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:11:21.851 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-17T12:11:21.851 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-12-17T12:11:21.851 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.8 2023-12-17T12:11:22.013 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:21 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:21.969+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:22.014 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:21 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:21.969+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:22.014 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:21 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:21.969+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:22.014 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:21 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:21.969+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:22.014 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:21 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:21.970+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:22.232 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:21 smithi192 systemd[1]: Stopping Ceph osd.8 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:11:22.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:21 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:21.950+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:22.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:21 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:21.950+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:22.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:21 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:21.950+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:22.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:21 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:21.950+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:22.544 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:22.232+0000 7f1c1212d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:11:22.544 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:22.232+0000 7f1c1212d700 -1 osd.8 87 *** Got signal Terminated *** 2023-12-17T12:11:22.544 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:22.232+0000 7f1c1212d700 -1 osd.8 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:11:22.545 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:22.544+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:22.810 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:22.544+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:22.810 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:22.544+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:22.810 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:22.544+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.728872+0000 front 2023-12-17T12:10:52.728848+0000 (oldest deadline 2023-12-17T12:11:18.628743+0000) 2023-12-17T12:11:22.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:22.671+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:22.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:22.671+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:22.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:22.671+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:22.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:22 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:22.671+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:23.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:22 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:22.948+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:23.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:22 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:22.948+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:23.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:22 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:22.948+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:23.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:22 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:22.948+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:23.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:22 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:22.948+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:23.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:22 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:22.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:23.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:22 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:22.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:23.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:22 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:22.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:23.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:22 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:22.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:23.810 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:23 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:23.516+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:23.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:23 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:23.516+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:23.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:23 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:23.516+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:23.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:23 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:23.516+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.728872+0000 front 2023-12-17T12:10:52.728848+0000 (oldest deadline 2023-12-17T12:11:18.628743+0000) 2023-12-17T12:11:23.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:23 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:23.654+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:23.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:23 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:23.654+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:23.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:23 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:23.654+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:23.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:23 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:23.654+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:24.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:23 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:23.980+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:24.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:23 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:23.980+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:24.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:23 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:23.980+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:24.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:23 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:23.980+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:24.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:23 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:23.980+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:24.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:23 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:23.956+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:24.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:23 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:23.956+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:24.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:23 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:23.956+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:24.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:23 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:23.956+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:25.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:24 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:24.561+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:25.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:24 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:24.561+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:25.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:24 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:24.561+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:25.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:24 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:24.561+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.728872+0000 front 2023-12-17T12:10:52.728848+0000 (oldest deadline 2023-12-17T12:11:18.628743+0000) 2023-12-17T12:11:25.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:24 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:24.628+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:25.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:24 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:24.628+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:25.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:24 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:24.628+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:25.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:24 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:24.628+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:25.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:24 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:24.944+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:25.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:24 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:24.944+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:25.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:24 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:24.944+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:25.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:24 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:24.944+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:25.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:24 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:24.944+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:25.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:24 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:24.944+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:25.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:24 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:24.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:25.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:24 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:24.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:25.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:24 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:24.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:25.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:24 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:24.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:25.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:25 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:25.634+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:25.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:25 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:25.634+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:25.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:25 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:25.634+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:25.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:25 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:25.634+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:25.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:25 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:25.558+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:25.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:25 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:25.558+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:25.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:25 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:25.558+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:25.811 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:25 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:25.558+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.728872+0000 front 2023-12-17T12:10:52.728848+0000 (oldest deadline 2023-12-17T12:11:18.628743+0000) 2023-12-17T12:11:26.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:25 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:25.946+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:26.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:25 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:25.946+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:26.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:25 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:25.946+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:26.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:25 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:25.946+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:26.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:25 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:25.946+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:26.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:25 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:25.946+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:26.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:25 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:25.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:26.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:25 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:25.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:26.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:25 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:25.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:26.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:25 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:25.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:26.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:25 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:25.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:27.060 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:26.587+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:28.824403+0000 front 2023-12-17T12:10:28.824494+0000 (oldest deadline 2023-12-17T12:10:54.124496+0000) 2023-12-17T12:11:27.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:26.587+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.625656+0000 front 2023-12-17T12:10:37.625748+0000 (oldest deadline 2023-12-17T12:11:02.325844+0000) 2023-12-17T12:11:27.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:26.587+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.828273+0000 front 2023-12-17T12:10:46.828402+0000 (oldest deadline 2023-12-17T12:11:12.728152+0000) 2023-12-17T12:11:27.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:26.587+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.728872+0000 front 2023-12-17T12:10:52.728848+0000 (oldest deadline 2023-12-17T12:11:18.628743+0000) 2023-12-17T12:11:27.061 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8[64645]: 2023-12-17T12:11:26.587+0000 7f1c0bf23700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:02.130091+0000 front 2023-12-17T12:11:02.130049+0000 (oldest deadline 2023-12-17T12:11:25.629989+0000) 2023-12-17T12:11:27.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:26.649+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:27.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:26.649+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:27.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:26.649+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:27.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:26 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:26.649+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:27.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:26 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:26.963+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:27.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:26 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:26.963+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:27.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:26 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:26.963+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:27.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:26 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:26.963+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:27.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:26 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:26.963+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:27.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:26 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:26.963+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:27.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:26 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:26.972+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:27.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:26 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:26.972+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:27.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:26 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:26.972+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:27.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:26 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:26.972+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:27.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:26 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:26.972+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:27.917 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:27 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:27.635+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:27.917 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:27 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:27.635+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:27.917 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:27 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:27.635+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:27.917 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:27 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:27.635+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:27.917 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:27 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:27.635+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:28.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:28.006+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:28.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:28.006+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:28.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:28.006+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:28.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:28.006+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:28.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:28.006+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:28.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:28 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:28.006+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:28.310 INFO:journalctl@ceph.osd.8.smithi192.stdout:Dec 17 12:11:27 smithi192 bash[81705]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-8 2023-12-17T12:11:28.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:27 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:27.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:28.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:27 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:27.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:28.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:27 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:27.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:28.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:27 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:27.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:28.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:27 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:27.923+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:29.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:28 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:28.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:29.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:28 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:28.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:29.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:28 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:28.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:29.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:28 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:28.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:29.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:28 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:28.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:29.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:28 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:28.926+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:29.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:28 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:28.926+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:29.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:28 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:28.926+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:29.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:28 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:28.926+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:29.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:28 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:28.926+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:29.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:29 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:29.042+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:29.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:29 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:29.042+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:29.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:29 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:29.042+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:29.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:29 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:29.042+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:29.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:29 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:29.042+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:29.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:29 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:29.042+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:29.581 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.8.service' 2023-12-17T12:11:29.669 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:11:29.670 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-12-17T12:11:29.670 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-12-17T12:11:29.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.9 2023-12-17T12:11:30.055 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:29 smithi027 systemd[1]: Stopping Ceph osd.9 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:11:30.055 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:29 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:29.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:30.055 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:29 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:29.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:30.055 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:29 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:29.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:30.055 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:29 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:29.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:30.056 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:29 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:29.924+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:30.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:29 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:29.626+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:30.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:29 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:29.626+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:30.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:29 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:29.626+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:30.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:29 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:29.626+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:30.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:29 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:29.626+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:30.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:30 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:30.054+0000 7f08f619e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:11:30.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:30 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:30.054+0000 7f08f619e700 -1 osd.9 87 *** Got signal Terminated *** 2023-12-17T12:11:30.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:30 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:30.054+0000 7f08f619e700 -1 osd.9 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:11:30.513 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.023+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:30.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.023+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:30.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.023+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:30.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.023+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:30.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.023+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:30.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.023+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:31.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:30.647+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:31.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:30.647+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:31.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:30.647+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:31.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:30.647+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:31.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:30.647+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:31.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:31.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:31.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:31.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:31.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:31.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:30 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:30.989+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:31.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:30 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:30.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:31.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:30 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:30.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:31.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:30 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:30.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:31.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:30 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:30.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:31.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:30 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:30.966+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:32.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:31 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:31.676+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:32.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:31 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:31.676+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:32.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:31 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:31.676+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:32.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:31 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:31.676+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:32.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:31 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:31.676+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:32.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:31 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:31.945+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:32.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:31 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:31.945+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:32.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:31 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:31.945+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:32.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:31 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:31.945+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:32.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:31 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:31.945+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:32.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:31 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:31.945+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:32.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:31.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:32.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:31.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:32.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:31.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:32.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:31.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:32.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:31 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:31.976+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:33.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:32.640+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:33.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:32.640+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:33.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:32.640+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:33.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:32.640+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:33.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:32.640+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:33.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:32 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:32.640+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:33.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:32 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:32.958+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:33.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:32 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:32.958+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:33.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:32 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:32.958+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:33.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:32 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:32.958+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:33.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:32 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:32.958+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:33.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:32 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:32.958+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:33.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:32 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:32.927+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:33.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:32 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:32.927+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:33.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:32 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:32.927+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:33.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:32 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:32.927+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:33.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:32 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:32.927+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:34.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:33.636+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:34.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:33.636+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:34.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:33.636+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:34.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:33.636+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:34.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:33.636+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:34.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:33 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:33.636+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:34.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:33.966+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:34.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:33.966+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:34.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:33.966+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:34.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:33.966+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:34.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:33.966+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:34.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:33.966+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:34.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:33 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:33.966+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:34.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:33.941+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:34.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:33.941+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:34.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:33.941+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:34.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:33.941+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:34.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:33 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:33.941+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:35.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:34.618+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:35.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:34.618+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:35.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:34.618+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:35.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:34.618+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:35.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:34.618+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:35.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:34.618+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:35.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:34 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:34.934+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:35.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:34 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:34.934+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:35.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:34 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:34.934+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:35.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:34 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:34.934+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:35.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:34 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:34.934+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:35.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:34 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:34.934+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:35.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:34 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:34.934+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:35.411 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:34 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:34.979+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:25.533328+0000 front 2023-12-17T12:10:25.533293+0000 (oldest deadline 2023-12-17T12:10:47.233400+0000) 2023-12-17T12:11:35.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:34 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:34.979+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:32.935890+0000 front 2023-12-17T12:10:32.936104+0000 (oldest deadline 2023-12-17T12:10:57.636151+0000) 2023-12-17T12:11:35.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:34 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:34.979+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:37.636896+0000 front 2023-12-17T12:10:37.637022+0000 (oldest deadline 2023-12-17T12:11:03.536781+0000) 2023-12-17T12:11:35.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:34 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:34.979+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:45.837993+0000 front 2023-12-17T12:10:45.838039+0000 (oldest deadline 2023-12-17T12:11:10.538127+0000) 2023-12-17T12:11:35.412 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:34 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9[80981]: 2023-12-17T12:11:34.979+0000 7f08f1f98700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.144307+0000 front 2023-12-17T12:11:04.144410+0000 (oldest deadline 2023-12-17T12:11:25.244442+0000) 2023-12-17T12:11:36.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:35.613+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:36.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:35.613+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:36.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:35.613+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:36.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:35.613+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:36.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:35.613+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:36.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:35 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:35.613+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:36.161 INFO:journalctl@ceph.osd.9.smithi027.stdout:Dec 17 12:11:35 smithi027 bash[101695]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-9 2023-12-17T12:11:36.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:35.886+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:36.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:35.886+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:36.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:35.886+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:36.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:35.886+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:36.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:35.886+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:36.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:35.886+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:36.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:35 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:35.886+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:37.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:36.658+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:37.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:36.658+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:37.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:36.658+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:37.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:36.658+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:37.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:36.658+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:37.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:36 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:36.658+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:37.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:36.931+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:37.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:36.931+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:37.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:36.931+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:37.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:36.931+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:37.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:36.931+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:37.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:36.931+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:37.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:36 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:36.931+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:37.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.9.service' 2023-12-17T12:11:37.533 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:11:37.534 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-12-17T12:11:37.534 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-12-17T12:11:37.534 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.10 2023-12-17T12:11:37.921 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 systemd[1]: Stopping Ceph osd.10 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:11:37.922 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.920+0000 7fc292c72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:11:38.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:37 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:37.642+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:38.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:37 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:37.642+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:38.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:37 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:37.642+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:38.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:37 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:37.642+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:38.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:37 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:37.642+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:38.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:37 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:37.642+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:38.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.920+0000 7fc292c72700 -1 osd.10 87 *** Got signal Terminated *** 2023-12-17T12:11:38.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.920+0000 7fc292c72700 -1 osd.10 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:11:38.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.970+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:38.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.970+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:38.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.970+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:38.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.970+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:38.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.970+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:38.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.970+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:38.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:37 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:37.970+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:39.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:38.606+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:39.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:38.606+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:39.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:38.606+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:39.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:38.606+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:39.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:38.606+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:39.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:38.606+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:39.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:38.997+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:39.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:38.997+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:39.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:38.997+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:39.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:38.997+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:39.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:38.997+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:39.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:38.997+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:39.265 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:38 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:38.997+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:40.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:39.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:40.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:39.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:40.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:39.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:40.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:39.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:40.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:39.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:40.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:39 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:39.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:40.263 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:40 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:40.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:40.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:40 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:40.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:40.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:40 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:40.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:40.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:40 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:40.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:40.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:40 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:40.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:40.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:40 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:40.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:40.264 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:40 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:40.003+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:41.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:40 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:40.597+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:41.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:40 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:40.597+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:41.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:40 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:40.597+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:41.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:40 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:40.597+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:41.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:40 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:40.597+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:41.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:40 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:40.597+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:41.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:41.014+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:41.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:41.014+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:41.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:41.014+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:41.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:41.014+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:41.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:41.014+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:41.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:41.014+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:41.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:41 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:41.014+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:42.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:41 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:41.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:42.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:41 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:41.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:42.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:41 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:41.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:42.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:41 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:41.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:42.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:41 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:41.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:42.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:41 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:41.569+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:42.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:42 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:42.063+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:22.654300+0000 front 2023-12-17T12:10:22.654352+0000 (oldest deadline 2023-12-17T12:10:50.854821+0000) 2023-12-17T12:11:42.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:42 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:42.063+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:30.855312+0000 front 2023-12-17T12:10:30.855239+0000 (oldest deadline 2023-12-17T12:10:56.755534+0000) 2023-12-17T12:11:42.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:42 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:42.063+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:36.756173+0000 front 2023-12-17T12:10:36.756254+0000 (oldest deadline 2023-12-17T12:11:06.756964+0000) 2023-12-17T12:11:42.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:42 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:42.063+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2023-12-17T12:10:46.757728+0000 front 2023-12-17T12:10:46.758002+0000 (oldest deadline 2023-12-17T12:11:12.057503+0000) 2023-12-17T12:11:42.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:42 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:42.063+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:52.057972+0000 front 2023-12-17T12:10:52.058007+0000 (oldest deadline 2023-12-17T12:11:20.858847+0000) 2023-12-17T12:11:42.514 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:42 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:42.063+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:01.360254+0000 front 2023-12-17T12:11:01.360239+0000 (oldest deadline 2023-12-17T12:11:24.860039+0000) 2023-12-17T12:11:42.515 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:42 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10[71213]: 2023-12-17T12:11:42.063+0000 7fc28f26d700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:08.663130+0000 front 2023-12-17T12:11:08.663026+0000 (oldest deadline 2023-12-17T12:11:33.962650+0000) 2023-12-17T12:11:42.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:42 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:42.555+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:42.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:42 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:42.555+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:42.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:42 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:42.555+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:42.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:42 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:42.555+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:42.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:42 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:42.555+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:42.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:42 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:42.555+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:43.763 INFO:journalctl@ceph.osd.10.smithi141.stdout:Dec 17 12:11:43 smithi141 bash[80332]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-10 2023-12-17T12:11:44.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:43 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:43.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:44.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:43 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:43.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:44.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:43 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:43.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:44.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:43 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:43.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:44.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:43 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:43.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:44.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:43 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:43.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:44.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:43 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:43.589+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6820 osd.7 since back 2023-12-17T12:11:16.733367+0000 front 2023-12-17T12:11:16.733307+0000 (oldest deadline 2023-12-17T12:11:42.633100+0000) 2023-12-17T12:11:44.946 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.10.service' 2023-12-17T12:11:45.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:11:45.032 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-12-17T12:11:45.033 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-12-17T12:11:45.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.11 2023-12-17T12:11:45.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:44 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:44.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:45.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:44 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:44.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:45.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:44 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:44.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:45.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:44 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:44.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:45.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:44 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:44.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:45.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:44 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:44.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:45.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:44 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:44.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6820 osd.7 since back 2023-12-17T12:11:16.733367+0000 front 2023-12-17T12:11:16.733307+0000 (oldest deadline 2023-12-17T12:11:42.633100+0000) 2023-12-17T12:11:45.420 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 systemd[1]: Stopping Ceph osd.11 for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:11:45.810 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.420+0000 7fe688693700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:11:45.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.420+0000 7fe688693700 -1 osd.11 87 *** Got signal Terminated *** 2023-12-17T12:11:45.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.420+0000 7fe688693700 -1 osd.11 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-17T12:11:45.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:45.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:45.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:45.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:45.811 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:45.812 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:45.812 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:45 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:45.622+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6820 osd.7 since back 2023-12-17T12:11:16.733367+0000 front 2023-12-17T12:11:16.733307+0000 (oldest deadline 2023-12-17T12:11:42.633100+0000) 2023-12-17T12:11:47.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:46.637+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:47.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:46.637+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:47.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:46.637+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:47.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:46.637+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:47.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:46.637+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:47.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:46.637+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:47.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:46 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:46.637+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6820 osd.7 since back 2023-12-17T12:11:16.733367+0000 front 2023-12-17T12:11:16.733307+0000 (oldest deadline 2023-12-17T12:11:42.633100+0000) 2023-12-17T12:11:48.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:47 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:47.625+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:48.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:47 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:47.625+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:48.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:47 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:47.625+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:48.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:47 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:47.625+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:48.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:47 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:47.625+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:48.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:47 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:47.625+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:48.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:47 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:47.625+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6820 osd.7 since back 2023-12-17T12:11:16.733367+0000 front 2023-12-17T12:11:16.733307+0000 (oldest deadline 2023-12-17T12:11:42.633100+0000) 2023-12-17T12:11:49.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:48.651+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:49.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:48.651+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:49.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:48.651+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:49.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:48.651+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:49.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:48.651+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:49.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:48.651+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:49.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:48 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:48.651+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6820 osd.7 since back 2023-12-17T12:11:16.733367+0000 front 2023-12-17T12:11:16.733307+0000 (oldest deadline 2023-12-17T12:11:42.633100+0000) 2023-12-17T12:11:50.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:49.607+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2023-12-17T12:10:20.422338+0000 front 2023-12-17T12:10:20.422224+0000 (oldest deadline 2023-12-17T12:10:46.322387+0000) 2023-12-17T12:11:50.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:49.607+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6804 osd.1 since back 2023-12-17T12:10:31.623799+0000 front 2023-12-17T12:10:31.623737+0000 (oldest deadline 2023-12-17T12:10:55.123552+0000) 2023-12-17T12:11:50.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:49.607+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-12-17T12:10:40.424677+0000 front 2023-12-17T12:10:40.424708+0000 (oldest deadline 2023-12-17T12:11:03.324879+0000) 2023-12-17T12:11:50.061 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:49.607+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6812 osd.4 since back 2023-12-17T12:10:51.828243+0000 front 2023-12-17T12:10:51.828148+0000 (oldest deadline 2023-12-17T12:11:17.727977+0000) 2023-12-17T12:11:50.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:49.607+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2023-12-17T12:11:04.030398+0000 front 2023-12-17T12:11:04.030350+0000 (oldest deadline 2023-12-17T12:11:26.930432+0000) 2023-12-17T12:11:50.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:49.607+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2023-12-17T12:11:11.532120+0000 front 2023-12-17T12:11:11.532195+0000 (oldest deadline 2023-12-17T12:11:32.632200+0000) 2023-12-17T12:11:50.062 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:49 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11[72547]: 2023-12-17T12:11:49.607+0000 7fe682c8a700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.141:6820 osd.7 since back 2023-12-17T12:11:16.733367+0000 front 2023-12-17T12:11:16.733307+0000 (oldest deadline 2023-12-17T12:11:42.633100+0000) 2023-12-17T12:11:51.310 INFO:journalctl@ceph.osd.11.smithi192.stdout:Dec 17 12:11:50 smithi192 bash[81959]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-osd-11 2023-12-17T12:11:52.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@osd.11.service' 2023-12-17T12:11:52.677 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:11:52.677 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-12-17T12:11:52.678 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-12-17T12:11:52.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.a 2023-12-17T12:11:53.077 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:11:52 smithi027 systemd[1]: Stopping Ceph mds.a for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:11:53.412 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:11:53 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-a[88721]: 2023-12-17T12:11:53.076+0000 7fd288944700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:11:53.413 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:11:53 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-a[88721]: 2023-12-17T12:11:53.076+0000 7fd288944700 -1 mds.a *** got signal Terminated *** 2023-12-17T12:11:57.661 INFO:journalctl@ceph.mds.a.smithi027.stdout:Dec 17 12:11:57 smithi027 bash[101951]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-a 2023-12-17T12:11:57.813 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.a.service' 2023-12-17T12:11:57.900 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:11:57.900 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-12-17T12:11:57.900 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-12-17T12:11:57.901 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.d 2023-12-17T12:11:58.161 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:11:57 smithi027 systemd[1]: Stopping Ceph mds.d for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:11:58.661 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:11:58 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-d[87726]: 2023-12-17T12:11:58.266+0000 7f0ec8858700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:11:58.661 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:11:58 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-d[87726]: 2023-12-17T12:11:58.266+0000 7f0ec8858700 -1 mds.d *** got signal Terminated *** 2023-12-17T12:12:02.860 INFO:journalctl@ceph.mds.d.smithi027.stdout:Dec 17 12:12:02 smithi027 bash[102089]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-d 2023-12-17T12:12:02.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.d.service' 2023-12-17T12:12:02.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:02.977 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-12-17T12:12:02.977 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-12-17T12:12:02.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.g 2023-12-17T12:12:03.161 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:12:03 smithi027 systemd[1]: Stopping Ceph mds.g for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:03.661 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:12:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-g[88338]: 2023-12-17T12:12:03.332+0000 7f61f1918700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:03.661 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:12:03 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-g[88338]: 2023-12-17T12:12:03.332+0000 7f61f1918700 -1 mds.g *** got signal Terminated *** 2023-12-17T12:12:06.912 INFO:journalctl@ceph.mds.g.smithi027.stdout:Dec 17 12:12:06 smithi027 bash[102227]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-g 2023-12-17T12:12:07.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.g.service' 2023-12-17T12:12:07.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:07.116 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-12-17T12:12:07.116 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-12-17T12:12:07.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.j 2023-12-17T12:12:07.411 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:12:07 smithi027 systemd[1]: Stopping Ceph mds.j for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:07.911 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:12:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-j[87341]: 2023-12-17T12:12:07.491+0000 7f330f7eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:07.911 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:12:07 smithi027 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-j[87341]: 2023-12-17T12:12:07.491+0000 7f330f7eb700 -1 mds.j *** got signal Terminated *** 2023-12-17T12:12:11.911 INFO:journalctl@ceph.mds.j.smithi027.stdout:Dec 17 12:12:11 smithi027 bash[102376]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-j 2023-12-17T12:12:12.148 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.j.service' 2023-12-17T12:12:12.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:12.232 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-12-17T12:12:12.232 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-12-17T12:12:12.233 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.b 2023-12-17T12:12:12.624 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:12:12 smithi141 systemd[1]: Stopping Ceph mds.b for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:13.013 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:12:12 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-b[77436]: 2023-12-17T12:12:12.621+0000 7f7bb3631700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:13.013 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:12:12 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-b[77436]: 2023-12-17T12:12:12.621+0000 7f7bb3631700 -1 mds.b *** got signal Terminated *** 2023-12-17T12:12:17.013 INFO:journalctl@ceph.mds.b.smithi141.stdout:Dec 17 12:12:16 smithi141 bash[80596]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-b 2023-12-17T12:12:17.210 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.b.service' 2023-12-17T12:12:17.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:17.294 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-12-17T12:12:17.295 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-12-17T12:12:17.295 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.e 2023-12-17T12:12:17.513 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:12:17 smithi141 systemd[1]: Stopping Ceph mds.e for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:18.013 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:12:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-e[76755]: 2023-12-17T12:12:17.663+0000 7f81df735700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:18.014 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:12:17 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-e[76755]: 2023-12-17T12:12:17.663+0000 7f81df735700 -1 mds.e *** got signal Terminated *** 2023-12-17T12:12:21.763 INFO:journalctl@ceph.mds.e.smithi141.stdout:Dec 17 12:12:21 smithi141 bash[80734]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-e 2023-12-17T12:12:21.954 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.e.service' 2023-12-17T12:12:22.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:22.037 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-12-17T12:12:22.037 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-12-17T12:12:22.037 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.h 2023-12-17T12:12:22.263 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:12:22 smithi141 systemd[1]: Stopping Ceph mds.h for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:22.763 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:12:22 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-h[77095]: 2023-12-17T12:12:22.420+0000 7f1fd4ad2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:22.764 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:12:22 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-h[77095]: 2023-12-17T12:12:22.420+0000 7f1fd4ad2700 -1 mds.h *** got signal Terminated *** 2023-12-17T12:12:25.763 INFO:journalctl@ceph.mds.h.smithi141.stdout:Dec 17 12:12:25 smithi141 bash[80874]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-h 2023-12-17T12:12:25.962 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.h.service' 2023-12-17T12:12:26.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:26.043 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-12-17T12:12:26.043 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-12-17T12:12:26.043 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.k 2023-12-17T12:12:26.264 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:12:26 smithi141 systemd[1]: Stopping Ceph mds.k for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:26.763 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:12:26 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-k[76415]: 2023-12-17T12:12:26.420+0000 7f9f6a7ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:26.764 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:12:26 smithi141 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-k[76415]: 2023-12-17T12:12:26.420+0000 7f9f6a7ab700 -1 mds.k *** got signal Terminated *** 2023-12-17T12:12:30.013 INFO:journalctl@ceph.mds.k.smithi141.stdout:Dec 17 12:12:29 smithi141 bash[81012]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-k 2023-12-17T12:12:30.128 DEBUG:teuthology.orchestra.run.smithi141:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.k.service' 2023-12-17T12:12:30.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:30.210 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-12-17T12:12:30.210 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-12-17T12:12:30.210 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.c 2023-12-17T12:12:30.560 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:12:30 smithi192 systemd[1]: Stopping Ceph mds.c for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:31.060 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:12:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-c[78620]: 2023-12-17T12:12:30.600+0000 7faa18f27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:31.061 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:12:30 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-c[78620]: 2023-12-17T12:12:30.600+0000 7faa18f27700 -1 mds.c *** got signal Terminated *** 2023-12-17T12:12:33.810 INFO:journalctl@ceph.mds.c.smithi192.stdout:Dec 17 12:12:33 smithi192 bash[82226]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-c 2023-12-17T12:12:34.041 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.c.service' 2023-12-17T12:12:34.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:34.124 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-12-17T12:12:34.124 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-12-17T12:12:34.124 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.f 2023-12-17T12:12:34.310 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:12:34 smithi192 systemd[1]: Stopping Ceph mds.f for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:34.810 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:12:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-f[78282]: 2023-12-17T12:12:34.478+0000 7f42dfd53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:34.810 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:12:34 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-f[78282]: 2023-12-17T12:12:34.478+0000 7f42dfd53700 -1 mds.f *** got signal Terminated *** 2023-12-17T12:12:37.810 INFO:journalctl@ceph.mds.f.smithi192.stdout:Dec 17 12:12:37 smithi192 bash[82368]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-f 2023-12-17T12:12:37.976 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.f.service' 2023-12-17T12:12:38.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:38.078 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-12-17T12:12:38.078 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-12-17T12:12:38.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.i 2023-12-17T12:12:38.310 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:12:38 smithi192 systemd[1]: Stopping Ceph mds.i for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:38.810 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:12:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-i[78962]: 2023-12-17T12:12:38.479+0000 7f056f7e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:38.810 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:12:38 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-i[78962]: 2023-12-17T12:12:38.479+0000 7f056f7e3700 -1 mds.i *** got signal Terminated *** 2023-12-17T12:12:41.810 INFO:journalctl@ceph.mds.i.smithi192.stdout:Dec 17 12:12:41 smithi192 bash[82506]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-i 2023-12-17T12:12:41.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.i.service' 2023-12-17T12:12:42.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:42.074 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-12-17T12:12:42.074 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-12-17T12:12:42.074 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.l 2023-12-17T12:12:42.310 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:12:42 smithi192 systemd[1]: Stopping Ceph mds.l for 62030760-9cd3-11ee-95a5-87774f69a715... 2023-12-17T12:12:42.810 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:12:42 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-l[77943]: 2023-12-17T12:12:42.443+0000 7f436282d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-17T12:12:42.810 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:12:42 smithi192 ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-l[77943]: 2023-12-17T12:12:42.443+0000 7f436282d700 -1 mds.l *** got signal Terminated *** 2023-12-17T12:12:47.060 INFO:journalctl@ceph.mds.l.smithi192.stdout:Dec 17 12:12:46 smithi192 bash[82645]: ceph-62030760-9cd3-11ee-95a5-87774f69a715-mds-l 2023-12-17T12:12:47.119 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62030760-9cd3-11ee-95a5-87774f69a715@mds.l.service' 2023-12-17T12:12:47.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-17T12:12:47.199 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-12-17T12:12:47.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62030760-9cd3-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-17T12:12:47.525 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:12:50.766 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62030760-9cd3-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-17T12:12:51.091 INFO:teuthology.orchestra.run.smithi141.stdout:Deleting cluster with fsid: 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:12:54.107 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62030760-9cd3-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-17T12:12:54.420 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:12:57.439 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:12:57.521 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:12:57.600 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-17T12:12:57.671 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-17T12:12:57.673 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi027/crash 2023-12-17T12:12:57.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash -- . 2023-12-17T12:12:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-17T12:12:57.758 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2023-12-17T12:12:57.761 DEBUG:teuthology.misc:Transferring archived files from smithi141:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi141/crash 2023-12-17T12:12:57.762 DEBUG:teuthology.orchestra.run.smithi141:> sudo tar c -f - -C /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash -- . 2023-12-17T12:12:57.841 INFO:teuthology.orchestra.run.smithi141.stderr:tar: /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-17T12:12:57.842 INFO:teuthology.orchestra.run.smithi141.stderr:tar: Error is not recoverable: exiting now 2023-12-17T12:12:57.845 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi192/crash 2023-12-17T12:12:57.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash -- . 2023-12-17T12:12:57.958 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/62030760-9cd3-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-17T12:12:57.958 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2023-12-17T12:12:57.961 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-17T12:12:57.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-12-17T12:12:58.046 INFO:teuthology.orchestra.run.smithi027.stderr:grep: /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph.log: No such file or directory 2023-12-17T12:12:58.051 INFO:tasks.cephadm:Compressing logs... 2023-12-17T12:12:58.052 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-17T12:12:58.056 DEBUG:teuthology.orchestra.run.smithi141:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-17T12:12:58.061 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-17T12:12:58.122 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-17T12:12:58.122 INFO:teuthology.orchestra.run.smithi027.stderr:: No such file or directory 2023-12-17T12:12:58.123 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.a.log 2023-12-17T12:12:58.124 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.g.log 2023-12-17T12:12:58.125 INFO:teuthology.orchestra.run.smithi141.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-17T12:12:58.125 INFO:teuthology.orchestra.run.smithi141.stderr:: No such file or directory 2023-12-17T12:12:58.126 INFO:teuthology.orchestra.run.smithi141.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.7.log 2023-12-17T12:12:58.126 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.j.log 2023-12-17T12:12:58.127 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.a.log 2023-12-17T12:12:58.127 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.1.log 2023-12-17T12:12:58.127 INFO:teuthology.orchestra.run.smithi141.stderr: 91.6% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-17T12:12:58.128 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.h.log 2023-12-17T12:12:58.129 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.g.log.gz 2023-12-17T12:12:58.129 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.9.log 2023-12-17T12:12:58.129 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.3.log 2023-12-17T12:12:58.129 INFO:teuthology.orchestra.run.smithi027.stderr: 94.6% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.j.log.gz 2023-12-17T12:12:58.131 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 : No such file or directory--verbose 2023-12-17T12:12:58.131 INFO:teuthology.orchestra.run.smithi192.stderr:-- /var/log/ceph/cephadm.log 2023-12-17T12:12:58.132 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.5.log 2023-12-17T12:12:58.132 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.i.log 2023-12-17T12:12:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.9.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-17T12:12:58.132 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log 2023-12-17T12:12:58.133 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log 2023-12-17T12:12:58.133 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.k.log 2023-12-17T12:12:58.133 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.e.log 2023-12-17T12:12:58.134 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.b.log 2023-12-17T12:12:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.3.log: 94.8% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.a.log.gz 2023-12-17T12:12:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.0.log 2023-12-17T12:12:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.d.log 2023-12-17T12:12:58.136 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.5.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-17T12:12:58.136 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.c.log 2023-12-17T12:12:58.136 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.f.log 2023-12-17T12:12:58.136 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.z.log 2023-12-17T12:12:58.137 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.11.log 2023-12-17T12:12:58.137 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.6.log 2023-12-17T12:12:58.138 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.e.log: /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.b.log: 94.7% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.k.log.gz 2023-12-17T12:12:58.138 INFO:teuthology.orchestra.run.smithi141.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.b.log 2023-12-17T12:12:58.139 INFO:teuthology.orchestra.run.smithi141.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.y.log 2023-12-17T12:12:58.139 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.2.log 2023-12-17T12:12:58.139 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.d.log: /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.6.log: 94.7% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.d.log.gz 2023-12-17T12:12:58.139 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.x.log 2023-12-17T12:12:58.141 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.b.log: /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.10.log 2023-12-17T12:12:58.142 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.11.log: /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log 2023-12-17T12:12:58.142 INFO:teuthology.orchestra.run.smithi192.stderr: 91.5% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.z.log.gz 2023-12-17T12:12:58.143 INFO:teuthology.orchestra.run.smithi141.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.4.log 2023-12-17T12:12:58.144 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.l.log 2023-12-17T12:12:58.144 INFO:teuthology.orchestra.run.smithi141.stderr: 91.1% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log.gz 2023-12-17T12:12:58.145 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.c.log 2023-12-17T12:12:58.145 INFO:teuthology.orchestra.run.smithi192.stderr: 91.5% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.i.log.gz 2023-12-17T12:12:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.x.log: 91.2% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log.gz 2023-12-17T12:12:58.149 INFO:teuthology.orchestra.run.smithi141.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.10.log: /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.4.log: 91.2% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.h.log.gz 2023-12-17T12:12:58.149 INFO:teuthology.orchestra.run.smithi141.stderr: 92.8% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.e.log.gz 2023-12-17T12:12:58.149 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.8.log 2023-12-17T12:12:58.150 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.c.log: 94.9% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.l.log.gz 2023-12-17T12:12:58.150 INFO:teuthology.orchestra.run.smithi192.stderr: 92.9% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.f.log.gz 2023-12-17T12:12:58.150 INFO:teuthology.orchestra.run.smithi141.stderr: 91.4% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.y.log.gz 2023-12-17T12:12:58.153 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.8.log: 92.5% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.c.log.gz 2023-12-17T12:12:58.158 INFO:teuthology.orchestra.run.smithi192.stderr: 91.1% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-volume.log.gz 2023-12-17T12:12:58.216 INFO:teuthology.orchestra.run.smithi141.stderr: 92.3% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mds.b.log.gz 2023-12-17T12:12:58.266 INFO:teuthology.orchestra.run.smithi192.stderr: 92.8% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.11.log.gz 2023-12-17T12:12:58.286 INFO:teuthology.orchestra.run.smithi192.stderr: 91.8% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.c.log.gz 2023-12-17T12:12:58.302 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mgr.x.log.gz 2023-12-17T12:12:58.328 INFO:teuthology.orchestra.run.smithi027.stderr: 92.2% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.3.log.gz 2023-12-17T12:12:58.343 INFO:teuthology.orchestra.run.smithi141.stderr: 91.9% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.b.log.gz 2023-12-17T12:12:58.351 INFO:teuthology.orchestra.run.smithi192.stderr: 92.0% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.5.log.gz 2023-12-17T12:12:58.355 INFO:teuthology.orchestra.run.smithi027.stderr: 92.2% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.0.log.gz 2023-12-17T12:12:58.362 INFO:teuthology.orchestra.run.smithi141.stderr: 92.2% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.4.log.gz 2023-12-17T12:12:58.363 INFO:teuthology.orchestra.run.smithi027.stderr: 92.3% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.9.log.gz 2023-12-17T12:12:58.373 INFO:teuthology.orchestra.run.smithi141.stderr: 92.3% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.10.log.gz 2023-12-17T12:12:58.381 INFO:teuthology.orchestra.run.smithi192.stderr: 92.3% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.8.log.gz 2023-12-17T12:12:58.390 INFO:teuthology.orchestra.run.smithi027.stderr: 92.1% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.6.log.gz 2023-12-17T12:12:58.395 INFO:teuthology.orchestra.run.smithi141.stderr: 92.2% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.7.log.gz 2023-12-17T12:12:58.423 INFO:teuthology.orchestra.run.smithi141.stderr: 92.0% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.1.log.gz 2023-12-17T12:12:58.428 INFO:teuthology.orchestra.run.smithi141.stderr: 2023-12-17T12:12:58.428 INFO:teuthology.orchestra.run.smithi141.stderr:real 0m0.330s 2023-12-17T12:12:58.428 INFO:teuthology.orchestra.run.smithi141.stderr:user 0m1.234s 2023-12-17T12:12:58.428 INFO:teuthology.orchestra.run.smithi141.stderr:sys 0m0.194s 2023-12-17T12:12:58.429 INFO:teuthology.orchestra.run.smithi192.stderr: 92.1% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-osd.2.log.gz 2023-12-17T12:12:58.431 INFO:teuthology.orchestra.run.smithi192.stderr: 2023-12-17T12:12:58.431 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m0.330s 2023-12-17T12:12:58.432 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m0.994s 2023-12-17T12:12:58.432 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.155s 2023-12-17T12:12:58.572 INFO:teuthology.orchestra.run.smithi027.stderr: 90.8% -- replaced with /var/log/ceph/62030760-9cd3-11ee-95a5-87774f69a715/ceph-mon.a.log.gz 2023-12-17T12:12:58.576 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-12-17T12:12:58.576 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m0.482s 2023-12-17T12:12:58.576 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m1.434s 2023-12-17T12:12:58.576 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.196s 2023-12-17T12:12:58.577 INFO:tasks.cephadm:Archiving logs... 2023-12-17T12:12:58.577 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi027/log 2023-12-17T12:12:58.578 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-17T12:12:58.838 DEBUG:teuthology.misc:Transferring archived files from smithi141:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi141/log 2023-12-17T12:12:58.839 DEBUG:teuthology.orchestra.run.smithi141:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-17T12:12:59.082 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi192/log 2023-12-17T12:12:59.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-17T12:12:59.306 INFO:tasks.cephadm:Removing cluster... 2023-12-17T12:12:59.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62030760-9cd3-11ee-95a5-87774f69a715 --force 2023-12-17T12:12:59.623 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:13:00.113 DEBUG:teuthology.orchestra.run.smithi141:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62030760-9cd3-11ee-95a5-87774f69a715 --force 2023-12-17T12:13:00.421 INFO:teuthology.orchestra.run.smithi141.stdout:Deleting cluster with fsid: 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:13:00.893 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62030760-9cd3-11ee-95a5-87774f69a715 --force 2023-12-17T12:13:01.205 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 62030760-9cd3-11ee-95a5-87774f69a715 2023-12-17T12:13:01.668 INFO:tasks.cephadm:Removing cephadm ... 2023-12-17T12:13:01.668 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-17T12:13:01.720 DEBUG:teuthology.orchestra.run.smithi141:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-17T12:13:01.765 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-17T12:13:01.809 INFO:tasks.cephadm:Teardown complete 2023-12-17T12:13:01.810 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-17T12:13:01.819 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-17T12:13:01.819 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-17T12:13:01.824 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-17T12:13:01.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-17T12:13:01.995 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-17T12:13:02.004 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-17T12:13:02.004 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-17T12:13:02.010 DEBUG:teuthology.orchestra.run.smithi141:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-17T12:13:02.015 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-17T12:13:02.049 INFO:teuthology.orchestra.run.smithi027.stderr:bash: ntpq: command not found 2023-12-17T12:13:02.052 INFO:teuthology.orchestra.run.smithi141.stderr:bash: ntpq: command not found 2023-12-17T12:13:02.056 INFO:teuthology.orchestra.run.smithi192.stderr:bash: ntpq: command not found 2023-12-17T12:13:02.058 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-17T12:13:02.058 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-12-17T12:13:02.058 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +1337us[+1337us] +/- 73ms 2023-12-17T12:13:02.058 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 +139us[ +163us] +/- 54ms 2023-12-17T12:13:02.058 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -2019us[-2019us] +/- 94ms 2023-12-17T12:13:02.058 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T12:13:02.059 INFO:teuthology.orchestra.run.smithi141.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-17T12:13:02.059 INFO:teuthology.orchestra.run.smithi141.stdout:=============================================================================== 2023-12-17T12:13:02.060 INFO:teuthology.orchestra.run.smithi141.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +1364us[+1364us] +/- 73ms 2023-12-17T12:13:02.060 INFO:teuthology.orchestra.run.smithi141.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 +163us[ +165us] +/- 54ms 2023-12-17T12:13:02.060 INFO:teuthology.orchestra.run.smithi141.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -1969us[-1969us] +/- 94ms 2023-12-17T12:13:02.060 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T12:13:02.064 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-17T12:13:02.064 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-12-17T12:13:02.064 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 +1360us[+1360us] +/- 73ms 2023-12-17T12:13:02.064 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 +161us[ +142us] +/- 54ms 2023-12-17T12:13:02.064 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -2017us[-2017us] +/- 94ms 2023-12-17T12:13:02.065 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-17T12:13:02.070 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-17T12:13:02.080 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-17T12:13:02.080 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-17T12:13:02.089 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-17T12:13:02.234 DEBUG:teuthology.orchestra.run.smithi141:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-17T12:13:02.358 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-17T12:13:02.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-17T12:13:02.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T12:13:02.558 DEBUG:teuthology.orchestra.run.smithi141:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-17T12:13:02.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T12:13:02.630 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-17T12:13:02.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T12:13:02.701 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-17T12:13:02.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-17T12:13:02.720 INFO:teuthology.task.internal:Duration was 1776.616097 seconds 2023-12-17T12:13:02.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-17T12:13:02.731 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-17T12:13:02.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-17T12:13:02.736 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-17T12:13:02.740 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-17T12:13:02.853 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-17T12:13:02.854 INFO:teuthology.orchestra.run.smithi141.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-17T12:13:02.857 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-17T12:13:03.191 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-17T12:13:03.191 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2023-12-17T12:13:03.191 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-17T12:13:03.253 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi141.front.sepia.ceph.com 2023-12-17T12:13:03.253 DEBUG:teuthology.orchestra.run.smithi141:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-17T12:13:03.312 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2023-12-17T12:13:03.312 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-17T12:13:03.370 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-17T12:13:03.371 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-17T12:13:03.376 DEBUG:teuthology.orchestra.run.smithi141:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-17T12:13:03.380 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-17T12:13:03.464 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-17T12:13:03.464 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-17T12:13:03.469 DEBUG:teuthology.orchestra.run.smithi141:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-17T12:13:03.474 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-17T12:13:03.602 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-17T12:13:03.612 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-17T12:13:03.612 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-17T12:13:03.683 DEBUG:teuthology.orchestra.run.smithi141:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-17T12:13:03.752 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-17T12:13:03.821 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-17T12:13:03.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-17T12:13:03.845 DEBUG:teuthology.orchestra.run.smithi141:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-17T12:13:03.849 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-17T12:13:03.910 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2023-12-17T12:13:03.911 INFO:teuthology.orchestra.run.smithi141.stdout:kernel.core_pattern = core 2023-12-17T12:13:03.916 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2023-12-17T12:13:03.966 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-17T12:13:04.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T12:13:04.008 DEBUG:teuthology.orchestra.run.smithi141:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-17T12:13:04.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T12:13:04.050 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-17T12:13:04.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-17T12:13:04.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-17T12:13:04.101 INFO:teuthology.task.internal:Transferring archived files... 2023-12-17T12:13:04.102 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi027 2023-12-17T12:13:04.102 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-17T12:13:04.182 DEBUG:teuthology.misc:Transferring archived files from smithi141:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi141 2023-12-17T12:13:04.182 DEBUG:teuthology.orchestra.run.smithi141:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-17T12:13:04.257 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156/remote/smithi192 2023-12-17T12:13:04.258 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-17T12:13:04.333 INFO:teuthology.task.internal:Removing archive directory... 2023-12-17T12:13:04.333 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-17T12:13:04.337 DEBUG:teuthology.orchestra.run.smithi141:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-17T12:13:04.342 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-17T12:13:04.385 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-17T12:13:04.397 INFO:teuthology.task.internal:Not uploading archives. 2023-12-17T12:13:04.397 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-17T12:13:04.408 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-17T12:13:04.408 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-17T12:13:04.422 DEBUG:teuthology.orchestra.run.smithi141:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-17T12:13:04.427 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-17T12:13:04.462 INFO:teuthology.orchestra.run.smithi027.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 17 12:13 /home/ubuntu/cephtest 2023-12-17T12:13:04.465 INFO:teuthology.orchestra.run.smithi141.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 17 12:13 /home/ubuntu/cephtest 2023-12-17T12:13:04.470 INFO:teuthology.orchestra.run.smithi192.stdout: 524317 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 17 12:13 /home/ubuntu/cephtest 2023-12-17T12:13:04.473 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-17T12:13:04.483 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-17T12:13:04.512 INFO:teuthology.nuke:Checking targets against current locks 2023-12-17T12:13:04.540 DEBUG:teuthology.nuke:shortname: smithi027 2023-12-17T12:13:04.540 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-17T12:13:04.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-17 11:21:20.059199', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJfU3CHyQpUsQAcUcyVukX5f4b233EJ0cgE5MeH0dQyPmvyV0T2YL2GB9Uh6UH/cMITD1X+OOZLl6TZrR/V9l4='} 2023-12-17T12:13:04.568 DEBUG:teuthology.nuke:shortname: smithi141 2023-12-17T12:13:04.568 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-17T12:13:04.593 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi141.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-17 11:21:20.058187', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOe8Da5pv1SEaTva5QR3wtOn+H7+J1X6wXZUHaF/ec9BRlBZnGoS14lnzlLQRXCyLk4pSvIsiug/Dz1EjWQEVg0='} 2023-12-17T12:13:04.594 DEBUG:teuthology.nuke:shortname: smithi192 2023-12-17T12:13:04.594 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-17T12:13:04.620 INFO:teuthology.orchestra.console.smithi027:Power off 2023-12-17T12:13:04.621 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-17T12:13:04.629 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-17_11:19:43-fs-rishabh-2023dec11-testing-default-smithi/7495156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-17 11:21:20.060050', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMO8E8LJa+RODPFtq6V0DO8UTBc3wGWc8fzkgc9tgZJQ2rcpwfOjAWCqFj7QqTSQFuD1xjwc9S3RUJZU2bHW8Q='} 2023-12-17T12:13:04.643 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2023-12-17T12:13:04.643 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:04.663 INFO:teuthology.orchestra.console.smithi141:Power off 2023-12-17T12:13:04.663 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-17T12:13:04.675 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-12-17T12:13:04.685 DEBUG:teuthology.orchestra.console.smithi141:power off output: Chassis Power Control: Down/Off 2023-12-17T12:13:04.685 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:04.704 INFO:teuthology.orchestra.console.smithi192:Power off 2023-12-17T12:13:04.704 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-17T12:13:04.712 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on 2023-12-17T12:13:04.725 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2023-12-17T12:13:04.725 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:04.745 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-12-17T12:13:08.676 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:08.714 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:08.747 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:08.786 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-12-17T12:13:08.822 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on 2023-12-17T12:13:08.858 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-12-17T12:13:12.787 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:12.823 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:12.859 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:12.897 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2023-12-17T12:13:12.930 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-12-17T12:13:12.967 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-12-17T12:13:12.997 INFO:teuthology.orchestra.console.smithi027:Power off completed 2023-12-17T12:13:13.031 INFO:teuthology.orchestra.console.smithi141:Power off completed 2023-12-17T12:13:16.972 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-17T12:13:17.083 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-12-17T12:13:17.184 INFO:teuthology.orchestra.console.smithi192:Power off completed 2023-12-17T12:13:17.290 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} duration: 1776.6160967350006 failure_reason: No module named 'tasks.fs' flavor: default mon.a-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c mon.b-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c mon.c-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=863a38a57a23417babca9d1a66fe6826 status: fail success: false 2023-12-17T12:13:17.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-17T12:13:17.367 INFO:teuthology.run:FAIL